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SUMMARY 


Since  1976,  the  Air  Force  Human  Resources  Laboratory  has  conducted 
research  and  development  (R&D)  to  develop  the  technology  to  present 
maintenance  technical  data  on  a  computer-based  system.  Several  major  efforts 
have  been  completed  Including  two  feasibility  studies,  the  development  of  two 
prototype  systems  to  support  Intermediate  level  maintenance,  and  the 
development  of  a  portable  computer  system  for  presentation  of  technical  data 
for  on-equipment  maintenance.  The  RM)  has  included  both  laboratory  studies  to 
develop  the  required  technologies  and  tests  of  the  prototype  systems  under 
realistic  field  conditions  In  order  to  ensure  that  the  systems  satisfactorily 
meet  the  needs  of  the  users. 

The  efforts  have  demonstrated  that  the  presentation  of  maintenance 
technical  data  on  a  computer-based  system  is  feasible  and  that  an  automated 
system  has  the  potential  to  improve  performance  and  reduce  the  costs  of 
maintaining  the  Air  Force  technical  order  system.  In  addition,  effective 
man/machine- interface  techniques,  data  presentation  techniques,  and  draft 
specifications  for  computer  hardware  and  software  were  developed.  The  results 
of  each  of  the  efforts  are  described,  and  recommendations  for  the  development 
of  automated  technical  data  presentation  systems  for  operational  use  are 
provided. 


Accession  For 


HTIS  GRA&I 

dtic  tab 

Unannounc ed 
Justlficatioi 

□ 

Distribution/ 

Availability  Codes 

Dlst 

Avail  < 
Spec' 

md/or 

al 

1 


PREFACE 


This  report  summarizes  R&D  accomplished  under  Project  2362, 
Computer-based  Maintenance  Aids  for  Technicians.  The  objectives  of 
the  research  were  to  develop  the  technology  to  present  technical  data 
on  an  automated  technical  data  system  and  to  develop  and  evaluate 
prototype  systems  for  intermediate-level  and  on-equipment 
maintenance.  The  work  built  upon  the  results  of  a  feasibility  study 
conducted  with  Laboratory  Director's  Funds,  which  is  also  described 
in  the  report.  The  work  was  sponsored  by  the  Logistics  and  Human 
Factors  Division  of  the  Air  Force  Human  Resources  Laboratory.  Both 
in-house  and  contractual  efforts  were  accomplished. 

The  report  describes  work  accomplished  under  five  work  units. 
The  work  units  and  key  Government  and  contractor  personnel  are  listed 
below: 

ILIR-00-34,  Human  Factors  Study  of  the  Feasibility  of  an 
Automated  Technical  Order  System.  Key  Government  Personnel:  Mr. 
John  J.  K.  Klesch  and  Ms.  Wendy  B.  Campbell.  Key  Contractor 

Personnel:  Dr.  Thomas  W.  Frazier  and  Mr.  Michael  K.  O'Heeron, 

Behavioral  Technology  Consultants,  Inc. 

2362-00-01,  Optimal  Formats  for  an  Automated  Technical  Order 
System.  Key  Government  Personnel:  Ms.  Wendy  B.  Campbell  and  Ms. 
Frances  A.  Greene.  Key  Contractor  Personnel:  Dr.  Thomas  W.  Frazier, 
Dr.  J.  Thomas  Roth,  and  Mr.  S.  Y.  Huang,  Behavioral  Technology 
Corporation. 

2362-00-02,  Development  and  Evaluation  of  a  Prototype 

Computer-based  Maintenance  Aids  System.  Key  Government  Personnel: 
Ms.  Wendy  B.  Campbell,  Ms.  Frances  A.  Greene,  and  Dr.  Donald  L. 
Thomas.  Key  Contractor  Personnel:  Mr.  Walter  Holmes,  and  Mr.  James 
Mitchell,  Unified  Industries,  Inc.;  and  Mr.  G.  Richard  Hatterick, 

BioTechnology,  Inc. 

2362-00-03,  Computer-based  Maintenance  Aids  System.  Key 
Government  Personnel:  Mr.  David  R.  Gunning,  Ms.  Barbara  L. 
Masquelier,  Dr.  Donald  L.  Thomas,  Lt  Jeffery  D.  Clay,  Major  Paul 
Condit,  and  Mr.  Timothy  Hansell.  Key  Contractor  Personnel:  Mr. 
Perry  Lanxner,  Rockwell  International;  Dr.  Alice  Agin,  Dr.  Richard 
Funk,  Ms.  Jane  Herman,  Ms.  Pat  Newlands,  Hughes  Aircraft  Company;  and 
Mr.  G.  Richard  Hatterick,  BioTechnology,  Inc. 

2362-00-04,  Portable  Computer-based  Maintenance  Aids  System. 
Key  Government  Personnel:  Capt  Stanley  J.  Collins,  Lt  Dean  Orrell, 
Ms.  Gall  A.  Hudson,  Mr.  Timothy  Hansell,  and  Capt  Michael  J.  Seus. 
Key  Contractor  Personnel:  Mr.  Roy  LeCrone,  Mr.  Mike  Darnold,  Mr. 
William  Hlgglnbottom,  and  Mr.  Steve  Werner,  Boeing  Military  Airplane 
Company. 


Many  Individuals  have  made  significant  contributions  to  the  work 
accomplished  under  the  project.  Overall  direction  was  provided  by 
Mr.  Robert  C.  Johnson,  Chief,  Combat  Logistics  Branch.  The  following 
personnel  have  served  as  Project  2362  Program  Manager  during  the 
11-year  history  of  the  project:  Mr.  John  J.  K.  Klesch,  Ms.  Wendy  B. 
Campbell,  Ms.  Frances  A.  Green,  Dr.  Donald  L.  Thomas,  Mr.  David  R. 
Gunning,  Capt  Stanley  J.  Collins,  Ms.  Barbara  L.  Masquelier,  and  Lt 
Jeffery  D.  Clay. 

In  addition  to  the  Government  personnel  listed  above,  several 
personnel  from  the  Strategic  Air  Command  have  made  significant 
contributions  to  the  project.  Particular  appreciation  is  expressed 
to: 

Major  Mike  Mull  and  CMSgt  Fred  Sterling,  HQ  SAC,  for  their 

assistance  in  arranging  for  technicians  to  participate  in  the  field 
tests. 

Personnel  of  the  55th  Reconnaissance  Wing,  Radar  Shop,  Offutt 
AFB,  NE,  for  their  assistance  In  the  field  test  of  the  first 

prototype  system. 

Personnel  of  the  305th  Air  Refueling  Wing,  Radar  Shop,  Grissom 
AFB,  IN,  for  their  assistance  In  the  field  test  of  the  first 

prototype  system. 
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COMPUTER-BASED  MAINTENANCE  AIDS  FOR  TECHNICIANS: 
PROJECT  FINAL  REPORT 


I.  INTRODUCTION 

Since  1976,  the  Air  Force  Human  Resources  Laboratory  (AFHRL)  has  conducted 
research  and  development  (R&D)  to  develop  the  technology  for  the  presentation 
of  technical  data  on  an  automated  system.  The  research  was  initiated  in 
recognition  of  the  potential  of  an  automated  technical  data  system  to  improve 
performance  of  maintenance  personnel  and  reduce  the  cost  of  maintaining  the 
Air  Force  technical  data  system.  The  emphasis  in  conducting  the  R&D  was 
placed  upon  designing  technical  data  presentation  techniques  and  procedures 
tailored  to  meet  the  technicians’  needs  and  make  it  easier  for  them  to  do 
their  job.  Emphasis  was  placed  upon  developing  data  access  techniques  which 
make  it  easy  to  locate  needed  information  and  developing  presentation  formats 
which  make  it  easy  for  them  to  use  the  information.  Experienced  maintenance 
personnel  from  operational  maintenance  units  were  involved  in  all  phases  of 
the  program  (as  consultants  and  test  subjects)  to  ensure  that  the  needs  of  the 
maintenance  technician  are  met  and  that  the  techniques  developed  are  suitable 
for  use  in  actual  maintenance  operations. 

A  laboratory  demonstration,  two  prototype  systems  for  intermediate  level 
maintenance,  and  a  prototype  system  for  on-equipment  maintenance  were 
developed.  Although  the  prototype  systems  were  not  intended  for  actual 
operational  implementation,  they  were  designed  to  fully  test  all  required 
functions  and  to  accurately  simulate  an  operational  system.  The  prototypes 
were  tested  in  maintenance  shops  of  operational  units  to  provide  realistic 
evaluations  of  the  systems  under  operational  conditions.  Specifications  for 
technical  data  content  and  system  hardware  and  software  for  use  in  developing 
systems  for  operational  use  were  developed  on  the  basis  of  knowledge  gained 
from  development  of  the  prototypes  and  from  experience  gained  in  the  field 
tests. 

The  majority  of  the  work  was  conducted  under  Project  2362,  Computer-based 
Maintenance  Aids  for  Technicians.  Five  major  efforts  and  several  smaller 
efforts  have  been  conducted  under  the  project.  The  results  of  these  efforts 
are  summarized  in  this  report. 


Background 

It  has  been  recognized  for  many  years  that  conventional  technical  orders 
(TOs)  used  to  support  maintenance  personnel  are  often  incomplete,  poorly 
organized,  and  difficult  to  use.  For  many  years,  AFHRL  conducted  an  R&D 
program  to  develop  better  technical  data  for  use  by  maintenance  personnel. 
The  emphasis  was  placed  upon  developing  technical  data  which  are  easy  to  use 
and  provide  In  one  place  all  of  the  Information  that  the  technician  needs. 
The  major  product  of  this  research  was  the  fully  procedural 1 zed  job 
performance  aid  (FPJPA).  FPJPAs  provided  the  technician  with  step-by-step 
Instructions  for  performing  assigned  tasks.  FPJPAs  are  based  upon  a  thorough 
task  analysis  to  ensure  that  the  procedures  are  complete,  accurate,  effective, 
and  suited  to  the  skills  of  the  Intended  user. 


Available  research  efforts  indicate  that  the  use  of  FPJPAs  can  result  in 
significant  improvements  in  performance  and  are  well  received  by  technicians. 
One  problem  associated  with  FPJPAs  is  that  they  require  many  more  pages  to 
cover  a  system  than  do  TOs.  This,  plus  the  increasing  complexity  of  modern 
aircraft  (and  the  resulting  increase  in  technical  data  requirements),  makes  it 
essential  that  a  more  cost-effective  method  of  maintaining  the  TO  system  be 
found. 

Automation  of  the  TO  system  appealed  to  be  the  logical  solution  to  the 
problem.  In  addition,  the  capabilities  offered  by  the  computer  appeared  to 
provide  the  potential  for  even  more  effective  technical  data  and  further 
enhancements  in  performance.  For  these  reasons.  Project  2362  was  initiated  in 
1976  to  develop  the  technology  for  the  presentation  of  technical  data  via  a 
computer  system  display.  At  approximately  the  same  time,  the  Automated 
Technical  Order  System  (ATOS)  project  was  initiated  by  the  Air  Force  Logistics 
Command  to  automate  the  production  and  updating  of  technical  data.  The 
long-range  plan  was  for  the  two  programs  to  converge  in  the  mid-1980s  so  that 
the  presentation  system  technology  developed  by  AFHRL  would  be  incorporated  as 
the  presentation  portion  of  the  ATOS  for  operational  application. 


Objective 

As  stated  in  the  Program  Management  Directive,  the  objective  of  Project 
2362  was  to: 

...develop  a  prototype  computer-based  technical  data  system 
to  facilitate  the  productivity  of  Air  Force  maintenance 
personnel.  The  system  will  provide  information  at  the  work  site 
to  guide  technicians'  performance.  Attention  will  be  given  to 
determining  the  basic  needs  of  technicians  for  information  and 
the  characteristics  of  a  hardware  and  software  system  to  provide 
that  information.  A  major  consideration  will  be  the  efficiency 
and  effectiveness  of  the  man-computer  (software  and  hardware) 
interaction.  Initial  efforts  will  involve  clarifying  the 
technicians'  needs  for  information  and  an  assessment  of  the 
hardware  and  software  to  meet  those  needs.  With  special 
attention  to  problems  of  interface,  utilization  and  acceptance,  a 
limited  prototype  system  will  be  developed  and  demonstrated/ 
evaluated. 

The  individual  R&D  efforts  conducted  to  achieve  the  project  objectives  are 
briefly  described  In  the  following  paragraphs.  The  efforts  are  described  in 
detail  in  Sections  II  through  VI.  Recommendations  for  future  automated 
technical  data  presentation  systems  (ATDPS)  and  for  further  research  are 
provided  in  Section  VII. 


Approach 

The  basic  approach  taken  to  achieve  the  project  objective  was  to  first 
conduct  feasibility  studies  to  establish  the  feasibility  of  an  ATDPS,  identify 
the  basic  features  which  should  be  provided  by  such  a  system,  develop  and 
evaluate  prototype  systems,  and  develop  draft  specifications  for  use  in 


developing  and  procuring  systems  for  operational  use.  Two  feasibility  studies 
were  accomplished  and  three  prototype  systems  were  developed.  Evaluations  of 
two  of  the  prototypes  were  performed.  An  evaluation  of  the  third  prototype  is 
scheduled  for  the  Spring  of  1988. 


Project  Results 


Feasibility  Studies 

Historically,  the  first  attempt  to  automate  the  presentation  of  technical 
data  by  AFHRL  was  accomplished  as  part  of  the  earlier  FPJPA  research  (Project 
1194).  The  objectives  of  this  effort  were  to  provide  a  means: 

1.  For  ready  retrieval  of  data  organized  in  tree  form  by  personnel  having 
little  or  no  experience  with  computers; 

2.  For  rapid  updating  of  data  from  a  remote  terminal; 

3.  For  batch  storage,  updating,  and  editing  of  data;  and 

4.  For  allowing  experimental  building  of  data  files  from  a  remote 
terminal  by  personnel  having  minimal  or  no  experience  with  computers. 

Existing  software  packages  with  the  capability  to  handle  data  with 
extensive  branching  were  adapted  for  the  study.  The  software  was  installed  on 
a  mainframe  computer  system  (COC  6600)  at  the  Aeronautical  Systems  Division 
Computer  Center.  Data  prepared  in  a  fully  procedural i zed  troubleshooting 
format  were  input  to  the  system.  The  system  was  then  tested  by  having 
personnel  with  limited  computer  experience  retrieve  data  from  the  system  using 
a  remote  terminal  (teletype).  Although  the  basic  goals  of  the  project  were 
achieved,  communication  with  the  computer  system  via  the  terminal  was  too  slow 
for  practical  use.  Also,  the  system  was  not  able  to  handle  graphics.  It  was 
concluded  that  the  basic  concept  of  computer  retrieval  and  presentation  of 
data  had  potential  but  that  further  developments  in  computer  hardware  and 
software  were  required  for  effective  presentation  of  technical  data  for 
operational  use. 

The  effort  is  described  in  detail  in  Colwell  (1971),  Colwell  and  Risk 
(1971),  and  Colwell,  Risk,  and  Reed  (1971). 

Two  feasibility  studies  were  conducted  to  systematically  examine  the 
feasibility  of  an  ATDPS  and  to  develop  a  preliminary  system  design.  The 
studies  were  conducted  under  contract  by  Behavioral  Technology  Consultants, 
Inc.  (F33615-77-C-0Q43)  and  Behavioral  Technology  Corporation  (Contract 
F33615-78-C-0030) J 


^Behavioral  Technology  Corporation  was  established  as  a  result  of  a 
reorganization  of  Behavioral  Technology  Consultants,  Inc.  The  ownership  and 
key  personnel  of  the  two  companies  were  the  same. 
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The  objectives  of  the  feasibility  studies  were  to  systematically  examine 
the  feasibility  of  creating  and  implementing  an  ATDPS,  analyze  and  establish 
the  requirements  for  information  to  be  presented  on  the  system,  establish 
human  factors  requirements  for  the  system,  develop  system  design  concepts  for 
the  system,  and  conduct  a  preliminary  test  of  the  design  concepts  and  features 
developed. 

The  feasibility  studies  determined  that  the  development  of  an  ADTPS  was 
feasible  and  that  the  development  of  such  a  system  would  have  significant 
potential  to  improve  Air  Force  maintenance  operations.  A  basic  system  design 
was  developed  and  given  a  preliminary  test  in  a  laboratory  environment.  Key 
system  requirements  proposed  included: 

1.  The  system  should  be  designed  for  ease  of  use.  It  should  provide  the 
user  with  the  capability  to  easily  recall  any  desired  information. 

2.  A  special  function  keyboard  should  be  provided  to  simplify  interaction 
with  the  system.  The  special  functions  and  keyboard  should  be  designed  to 
permit  recall  of  any  desired  information  with  one  or  two  keystrokes. 

3.  Procedural  data  should  be  prepared  in  a  step-by-step  format  similar  to 
that  used  for  FPJPAs.  The  data  should  be  provided  in  multiple  levels  of 
detail  so  that  the  user  may  select  data  designed  to  be  compatible  with  his 
level  of  experience  and  training. 

4.  Easy  access  should  be  provided  to  “pools"  of  support  information  that 
the  user  may  want  to  refer  to  while  performing  a  task.  Pool  information  may 
include  system  information  (such  as  theory  of  operation,  system  descriptions, 
and  parts  information)  or  other  support  information  (such  as  how  to  set  up 
test  equipment). 

The  results  of  the  Behavioral  Technology  feasibility  studies  are 
summarized  in  Section  II. 


Initial  Prototype  Development 

A  contract  was  awarded  to  Unified  Industries,  Inc.  (UII)  In  September  1978 
to  develop  and  evaluate  a  full-scale  prototype  ATDPS  for  intermediate  level 
maintenance.  Initial  work  on  this  effort  focused  upon  the  development  of  the 
human  factors  and  technical  data  requirements  for  the  system.  Man/machine 
interface  (MMI)  techniques  (Including  a  function  keyboard  design)  and  data 
presentation  formats  were  developed.  Work  was  initiated  on  the  design  of  the 
hardware  and  software.  However,  It  became  necessary  to  terminate  the  contract 
before  the  system  could  actually  be  built.  The  decision  to  terminate  the 
contract  was  made  as  a  result  of  the  addition  of  a  requirement  by  the  sponsor 
that  the  system  must  be  deployable.  The  prototype  system  under  development 
was  designed  for  stationary  use.  Extensive  modifications  would  have  been 
necessary  to  make  it  deployable.  It  was  determined  to  be  In  the  best 
Interests  of  the  Government  to  terminate  the  contract. 

Although  the  prototype  was  not  completed,  significant  progress  was  made  In 
defining  the  requirements  for  an  ATDPS.  The  results  of  the  work  performed  are 
discussed  In  detail  In  Section  III. 
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Computer-based  Maintenance  Aids  System  (Q1AS)  Development  and  Evaluation 

A  second  effort  to  develop  an  ATDPS  prototype  for  intermediate  level 
maintenance  was  established  in  1982.  A  contract  was  awarded  to  Rockwell 
International  (prime  contractor)  and  Hughes  Aircraft  (subcontractor)  for  the 
development  of  the  Computer-based  Maintenance  Aids  System  (known  as  CMAS  I). 
Emphasis  was  placed  upon  developing  human  factors  and  data  presentation 
requirements.  A  decision  was  made  to  use  existing  hardware  and  to  adapt 
existing  software  for  the  effort,  rather  than  procure/develop  hardware  and 
software  specifically  designed  for  the  CMAS  I.  An  analysis  was  made  to 
determine  requirements  for  a  deployable  system;  however,  no  attempt  was  made 
to  make  the  CMAS  deployable. 

The  development  of  the  CMAS  I  was  based  upon  the  earlier  work  accomplished 
in  the  feasibility  studies  and  the  UII  effort.  In  addition,  further  analyses 
of  technician  information  needs  were  made,  and  three  design  studies  were 
accomplished  to  evaluate  several  design  issues.  Software  original ly  developed 
for  the  Navy  Technical  Information  Presentation  System  (NTIPS)  was  adapted  and 
extended  to  meet  the  needs  of  the  CMAS  I.  An  available  MODCOMP  Model  7840 
minicomputer  system  with  a  Rastertech  color  graphics  terminal  was  used  to  host 
the  CMAS. 

The  CMAS  I  was  installed  in  an  intermediate  level  avionics  maintenance 
shop  (Radar)  at  Offutt  AFB,  Nebraska,  for  evaluation.  The  evaluation  revealed 
that  the  CMAS  I  did  not  successfully  meet  the  goals  of  the  program.  The 
system  failed  to  achieve  acceptance  by  the  users.  The  users'  rejection  of  the 
system  was  primarily  due  to  the  extremely  slow  response  time  of  the  system 
(approximately  11  seconds  to  retrieve  a  typical  procedural  frame)  and  to 
several  ineffective  MMI  techniques  employed.  Although  the  CMAS  I  effort  was 
not  a  complete  success,  it  did  provide  very  valuable  information  for  the 
development  of  the  follow-on  system. 

The  CMAS  I  effort  is  described  in  detail  in  Section  IV. 


CMAS  II  Development  and  Evaluation 

The  CMAS  II  effort  was  Initiated  to  develop  an  ATDPS  for  intermediate 
level  maintenance  which  did  not  have  the  weaknesses  of  the  CMAS  I  and  would  be 
accepted  and  used  by  maintenance  technicians.  The  CMAS  II  was  developed 
in-house  using  available  off-the-shelf  hardware.  Extensive  modifications  to 
existing  software  were  made  to  provide  the  capabilities  required  for  the 
system. 

The  Grid  Compass  Model  1139  microcomputer  was  selected  as  the  hardware  for 
the  system.  The  CMAS  II  design  Incorporated  many  of  the  design  features 
provided  in  the  CMAS  I.  However,  in  developing  the  system,  emphasis  was 
placed  upon  improving  the  system  response  times,  eliminating  the  cumbersome 
MMI  techniques,  and  adding  additional  features  to  Improve  usability. 

Technical  data  for  a  testbed  system  (the  RT-728A  receiver  of  the  AN/APX-64, 
Identify  Friend  or  Foe  System)  were  reformatted  and  expanded.  The  data  were 
then  input  to  the  computer,  and  the  CMAS  II  was  taken  to  Grissom  AFB,  Indiana, 
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for  a  field  demonstration  in  June  1985.  The  CMAS  II  was  tested  by  having 
experienced  and  inexperienced  technicians  use  it  to  perform  representative 
maintenance  tasks.  The  tests  demonstrated  that  the  technicians  could 
effectively  perform  maintenance  using  the  system.  Questionnaires  and 
interviews  indicated  that  the  technicians  liked  the  system  and  would  like  to 
see  a  similar  system  implemented  for  operational  use.  In  addition  to  the 
Grissom  test,  a  joint  Air  Force/Navy  evaluation  of  the  CMAS  II  was  conducted 
by  the  Navy  Personnel  Research  and  Development  Center  (NPRDC).  The  system  was 
evaluated  using  Air  Force,  Navy,  and  Marine  technicians.  The  results  were 
similar  to  those  obtained  at  Grissom  AFB.  Following  the  CMAS  II 
demonstration,  system  functional  specifications  for  automated  technical  data 
systems  and  for  technical  data  to  be  presented  on  the  system  were  developed. 
The  specifications  were  based  primarily  upon  the  CMAS  II. 

The  CMAS  II  effort  is  described  in  detail  in  Section  V. 


Portable  Computer-based  Maintenance  Aids  System  Development 

The  next  step  in  the  program  was  to  extend  the  CMAS  technology  for 
on-equipment  (flightline)  maintenance.  A  small,  rugged,  lightweight  computer 
system  is  needed  for  use  on  the  flight line.  A  suitable  portable  computer 
system  did  not  exist.  Therefore,  a  contract  was  awarded  to  develop  a  portable 
computer-based  maintenance  aids  system  (PCMAS)  designed  specifically  for  the 
presentation  of  technical  data  for  on-equipment  maintenance.  The  contract  was 
awarded  to  Boeing  Military  Airplane  Company,  Huntsville,  Alabama,  on  31  July 
1985,  to  develop  the  system. 

The  design  and  fabrication  of  the  initial  PCMAS  unit  have  been  completed. 
The  PCMAS  is  a  semi -rugged i zed  portable  computer  designed  for  flightline  use. 
It  weighs  13  pounds  and  has  dimensions  of  12  x  15  x  3  inches.  Technical  data 
for  presentation  on  the  system  are  maintained  in  removable,  1-megabyte  memory 
cartridges.  The  PCMAS  Is  capable  of  operating  on  power  from  a  battery  pack, 
on  power  from  the  aircraft,  on  power  from  a  ground  power  unit,  or  on  standard 
commercial  power  (110  VAC).  A  keypad  with  eight  programmable  function  keys,  a 
numeric  keypad,  and  cursor  control  keys  are  provided  for  interaction  with  the 
system.  A  voice  activation  capability  is  also  provided.  A  built-in  1553  data 
bus  board  provides  a  capability  to  communicate  directly  with  aircraft  systems 
on  aircraft  equipped  with  a  MIL-STD-1 553  data  bus.  The  PCMAS  may  be  operated 
in  a  stand-alone  mode  or  in  conjunction  with  a  workstation  which  provides  a 
full  keyboard,  hard  disk  drive,  printer,  and  additional  communication 
capabilities.  Software  developed  for  the  system  Includes  a  UNIX-based 
operating  system  and  applications  software  for  the  presentation  of  technical 
data  and  for  computer-based  diagnostics. 

Three  prototype  units  were  produced  by  Boeing.  The  units  are  presently 
being  used  in-house  to  develop  additional  applications  and  to  modify  the 
production  software  to  correct  some  deficiencies.  Starting  In  the  spring  of 
1988,  studies  will  be  conducted  to  evaluate  the  PCMAS  as  a  technical  data 
presentation  system,  as  a  computer-based  diagnostics  aid,  and  as  a  device  for 
presenting  specialized  technical  data  for  aircraft  battle  damage  assessment 
(ABDA). 

Section  VI  presents  a  detailed  description  of  the  PCMAS. 
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Recommendations  and  Research  Needs 


The  research  described  above  provided  a  firm  basis  to  develop  recommenda¬ 
tions  for  the  development  of  automated  technical  data  systems  for  operational 
use.  Also,  in  evaluating  the  results  of  the  studies,  a  number  of  issues  have 
been  identified  which  require  further  research.  The  recommendations  developed 
and  the  research  needs  identified  are  discussed  in  Section  VII. 


Future  Efforts 

Although  the  Computer-based  Maintenance  Aids  for  Technicians  project  has 
officially  ended,  the  Laboratory  is  continuing  work  in  the  automation  of 
technical  data  and  other  types  of  information  used  by  maintenance  personnel. 
This  work  is  being  conducted  under  Project  2950,  Integrated  Maintenance 
Information  System  (IMIS).  The  IMIS  program  will  include  further  development 
of  techniques  for  presentation  of  technical  data  on  an  automated  technical 
data  system  and  the  development  of  techniques  for  the  automated  creation  of 
diagnostic  routines.  These  techniques  will  be  used  in  the  development  of  the 
IMIS,  which  will  integrate  the  following  information  systems:  technical  data 
presentation  systems,  automated  diagnostics  systems,  automated  maintenance 
management  systems,  computer-based  training  systems,  and  supply  systems.  The 
IMIS  will  make  it  possible  for  a  technician  to  access  any  information  required 
to  do  his  job  from  one  computer  system  using  a  common  set  of  data  access 
protocols.  The  program  is  described  in  Link,  Von  Holle,  and  Mason  (1987). 


II.  FEASIBILITY  STUDIES 

By  the  mid-1970s,  computer  technology  had  advanced  to  the  point  that  the 
use  of  computers  for  the  storage  and  presentation  of  technical  data  had  become 
feasible  and  practical.  Work  was  Initiated  to  establish  the  feasibility  of  an 
automated  technical  data  presentation  system  and  to  develop  a  preliminary 
system  design. 

The  initial  contract  in  the  CMAS  program  was  awarded  to  Behavioral 
Technology  Consultants,  Inc.  In  1976  for  a  study  to  examine  the  feasibility  of 
a  computer-based  technical  data  system,  to  identify  the  human  factors 
requirements  for  such  a  system,  and  to  develop  initial  system  design  con¬ 
cepts.  A  follow-on  contract  was  awarded  to  Behavioral  Technology  Corporation 
in  1977  to  further  develop  the  concepts.  The  overall  goals  of  this  work  were 
to  establish  the  feasibility  of  a  CMAS  and  to  conduct  an  Initial  "top-down" 
design  of  the  system.  The  basic  orientation  of  the  system  design  process  was 
to  design  the  system  from  the  viewpoint  of  the  maintenance  technician  to 
ensure  that  it  would  fully  support  the  user  in  the  work  place.  The  results  of 
these  studies  are  summarized  In  this  section.  The  materials  presented  in  this 
section  are  adapted  from  unpublished  reports  by  Frazier,  Campbell,  and  Kniess 
(1979)  and  Frazier,  Huang,  and  Roth  (1978). 


Objectives 

The  objectives  of  the  studies  were: 


1.  To  systematically  examine  the  feasibility  of  creating  and  implementing 
a  CMAS. 

2.  To  analyze  and  establish  the  requirements  for  technical  information  to 
be  presented  on  a  CMAS. 

3.  To  establish  the  human  factors  requirements  for  a  CMAS  system. 

4.  To  develop  system  design  concepts  for  presentation  of  technical 
information  on  a  computer  system  to  ensure  technician  acceptance,  enhanced 
performance,  and  usability. 

5.  To  conduct  a  preliminary  test,  in  a  laboratory  environment,  of  the 
design  concepts  and  features  developed. 


Approach 

A  four-phase  effort  was  conducted  to  achieve  the  above  objectives. 


Phase  1.  Human  Factors  Requirements 

The  purpose  of  this  phase  was  to  examine  the  human  factors  requirements 
for  a  computer-based  maintenance  aiding  system.  This  was  accomplished  by 
reviewing  available  MMI  techniques,  investigating  previous  research  on 
Improved  technical  data,  reviewing  relevant  literature,  and  studying  Air  Force 
maintenance  operations.  As  a  result  of  this  analysis,  four  specific  problem 
areas  were  identified  for  detailed  study.  These  were:  (a)  communication 
between  the  technician  and  the  system;  (b)  content  of  textual  materials;  (c) 
content  and  management  of  stationary  and  animated  graphics;  and  (d)  formats 
for  presentation  of  technical  data.  Each  of  these  areas  was  systematically 
studied.  Potential  approaches  and  solutions  were  developed,  refined,  and 
Incorporated  into  a  preliminary  system  design  for  test  on  a  laboratory  system. 


Phase  II.  Demonstrate  Feasibility 

The  purpose  of  the  second  phase  of  the  study  was  to  demonstrate  the 
feasibility  of  a  computer-based  maintenance  aiding  system  and  to  test  the 
preliminary  system  design  In  a  laboratory  environment.  A  small  minicomputer 
system  with  the  basic  capabilities  required  to  support  a  system  for  presenting 
automated  technical  data  was  assembled  at  the  contractor's  facility,  and  the 
necessary  software  was  developed  or  procured.  Technical  data  for  representa¬ 
tive  tasks  were  entered  in  the  system  to  provide  a  basis  for  evaluating  the 
various  MMI  and  data  presentation  techniques  developed  earlier. 


Phase  III.  Evaluate  Preliminary  System 

In  this  phase,  additional  technical  data  were  developed  for  use  In  a  more 
systematic  evaluation  of  the  system.  The  system  was  then  tested  using 
experienced  technicians  as  subjects. 


8 


Phase  IV.  Develop  System  Design  Requirements 

In  the  final  phase  of  the  effort,  the  Information  gathered  was  synthe¬ 
sized  to  develop  fundamental  system  design  requirements  and  preliminary 
hardware  and  software  specifications  for  a  CMAS. 


Findings 

Phase  I.  Human  Factors  Requirements 

The  findings  of  Phase  I  are  summarized  below. 

Criteria.  The  primary  criteria,  established  at  the  start  of  the  study, 
were  that  the  computer-based  maintenance  aiding  system  must  be  easy  to  use  and 
must  effectively  meet  all  of  the  technician's  needs  for  information  to  do  his 
job.  The  first  task  In  this  phase  was  to  further  define  the  usability 
criteria.  The  criteria  developed  provided  the  basis  for  evaluating  all  MMI 
techniques,  data  presentation  techniques,  and  design  considerations  developed 
during  the  course  of  the  study.  Seven  criteria  were  Identified.  The  first 
five  criteria  were  adapted  from  Chenzoff  (1973).  The  last  two  were  added  by 
the  researchers  for  this  effort.  The  seven  criteria  were: 

1.  Accuracy  -  Data  must  be  technically  correct. 

2.  Understandablllty  -  Data  must  be  easily  understood  by  the  user. 

3.  Retrievabllity  -  Required  data  must  be  easy  to  find. 

4.  Relevance  -  Data  must  meet  user  needs. 

5.  Completeness  -  Data  must  provide  a  technician  with  all  of  the 
Information  that  he  needs. 

6.  Portability  -  Data  must  be  easy  to  carry  and  use. 

7.  Availability  -  Current  data  must  be>  readily  available  in  the  work  area. 

An  analysis  was  made  of  the  problems  encountered  In  meeting  the  above 
requirements  using  paper-based  systems  and  the  potential  for  a  computer-based 
system  to  overcome  these  difficulties 

Man/Computer  Dialog  Design.  The  problem  of  how  the  technician  interacts 
with  the  computer  was  analyzed.  Two  primary  concerns  were  addressed:  the 
nature  of  the  man/machine  Interaction  and  the  method  used  by  the  technician  to 
enter  the  request  to  the  system.  With  regard  to  the  nature  of  Interaction, 
system  communication  should  be  as  natural  as  possible  (l.e.,  similar  to  normal 
communication  between  humans)  and  should  be  positively  reinforcing.  Analysis 
of  different  user  entry  requests  showed  that  requiring  the  technician  to  type 
In  all  necessary  Information  would  not  be  suitable  since  It  Is  t Ime-consuml ng 
and  requires  typing  skills  that  the  average  technician  does  not  have.  There¬ 
fore,  an  alternate  approach  was  developed  which  would  let  the  technician 
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quickly  enter  the  request  without  requiring  special  skills.  The  proposed 
solution  used  a  special  function  panel— a  "Dialog  Generator  Panel." 


The  proposed  Dialog  Generator  Panel  design  provides  23  special  function 
keys  for  entering  requests  to  the  computer.  Use  of  the  function  keys  allows 
the  technician  to  enter  a  request  with  one  keystroke  in  most  cases  and  only  a 
few  keystrokes  in  others.  Two  types  of  functions  are  provided: 

1.  Non-referenced  communication  functions  require  only  one  keystroke.  For 
example,  the  key  OUTLINE  SYSTEM  FUNCTION  can  be  used  to  retrieve  a  functional 
description  of  the  system  currently  being  maintained. 

2.  A  referenced  communication  function  requires  the  technician  to  press 
the  key  and  a  number  indicating  a  selection.  For  example,  pressing  DEFINE  and 
"1"  would  result  in  the  system  displaying  an  object  presented  on  the  screen 
and  identified  as  "1." 

The  proposed  special  functions  and  their  definitions  are  presented  in 
Table  1. 


Table  1. 

Proposed  Function  Keys 

Function 

mumam 

Description 

CORRECT  ENTRY 

N 

Used  to  correct  alphanumeric  data  entry 
that  Is  to  be  logged  in  the  system  or 
used  for  retrieval  of  information 
choices  from  subsystem  memory. 

LIST  FORMATS 

N 

Used  for  permitting  the  user  to  identify 
a  choice  among  alternative  types  of 
information  when  menus  are  provided  on 
the  cathode-ray  tube  (CRT)  screen. 

STEP  BACK 

N 

Used  to  retrieve  the  frame  that  appeared 
immediately  prior. 

OUTLINE  TASK 

N 

Used  to  obtain  a  summary  of  the  mainte¬ 
nance  task  at  hand  for  preview 
purposes,  prior  to  undertaking  it. 

OUTLINE  SYSTEM 

N 

Permits  retrieval  of  the  functional  data 
concerning  the  system/subsystem/ 
equipment  being  worked  upon. 

DESCRIBE 

R 

Used  when  graphic  portrayals  are 
requested  for  such  objects  as  parts, 
tools,  subassemblies,  etc. 

OPERATE 

R 

Used  for  retrieving  pseudoanimation 
sequences  or  textual,  descriptions  of 
tool  use. 

Table  1.  (Continued) 


function 

Type^ 

Description 

COMPLETE 

N 

Used  to  signify  completion  of  task  steps 
and  completion  of  the  overall  task. 

DEFINE 

R 

Used  to  retrieve  definitions  of  special 
technical  terms. 

EXPLAIN 

R 

Used  to  obtain  a  verbal  explanation 
where  one  is  likely  to  be  needed. 

WHAT  NEXT 

N 

Used  to  retrieve  follow-on  information 
whether  expressed  as  a  task  step  or  in 
some  other  manner. 

READY 

N 

Signals  that  the  user  is  prepared  to 
accept  new  information  presented  in 
sequential  form. 

LOCATE 

R 

Used  to  locate  schematic  descriptions 
and  illustrated  parts  breakdowns  where 
a  specific  item  must  be  found  or 
identified  from  a  larger  assembly  of 
items. 

YES 

N 

Used  when  the  system  is  required  to  make 
a  check  requiring  either  a  "yes"  or 
"no"  response. 

NO 

N 

Used  when  the  system  is  required  to  make 
a  check  requiring  either  a  "yes"  or 
"no"  response. 

SHOW  HOW 

R 

Used  to  invoke  a  pseudoanimation 
procedure  other  than  one  that 
demonstrates  tool  use. 

SIGNIFICANCE 

R 

Used  to  learn  about  crew  or  aircraft 
significance  of  a  given  fault  or  repair, 

UNDERSTAND 

N 

Used  to  convey  comprehension  of  frame 
contents  to  the  system. 

I  WOULD  LIKE 

R 

Used  to  communicate  choices  to  the 
system. 

CHECK 

N 

Used  to  Identify  the  presence  of  check¬ 
list  Items  or  compliance  with  procedure; 
listed  in  checklist  form. 

ZOOM 


N 


Permits  scaling  of  screen  contents 
to  a  desired  size. 


unction 


Table  1.  (Concluded) 


Description 


ROTATE/PERSPECTIVE 

N 

Used  to  obtain  different  view  angles  of 
a  component,  equipment,  or  other 
physical  object. 

COMPARE 

R 

Used  to  retrieve  deductive  materials 
that  the  user  wants  to  have  displayed 
simultaneously  on  the  CRT  screen. 

aN  Indicates  non-referenced  communication;  R  indicates  referenced 

communication. 

Backlighting  the  keyboard  was  recommended  so  that  only  those  keys 
currently  active  are  lit.  This  approach  was  proposed  in  order  to  lessen  the 
search  time  required  to  locate  the  desired  function  key. 

Supervisor-Technician  Discourse.  The  researchers  believed  that  the  imper¬ 
sonal,  directive  language  used  in  conventional  technical  orders  would  have  a 
negative  impact  on  the  technician's  motivation  to  use  the  system.  For  this 
reason,  it  was  recommended  that  the  language  used  to  state  instructions 
presented  on  the  system  be  similar  to  that  used  by  a  supervisor  telling  a 
technician  how  to  do  the  task.  An  analysis  was  made  of  the  language  and 
phrasing  of  instructions  used  by  supervisors  in  telling  how  to  do  a  task. 
Recordings  were  made  of  communications  between  Air  Force  maintenance  super¬ 
visors  and  technicians  performing  representative  tasks.  Analysis  of  the 
communications  yielded  a  style  of  communications  which  could  be  used  for 
presenting  computerized  technical  data  in  a  more  "natural"  fashion  than  used 
in  technical  orders.  For  example,  the  technical  order  might  say,  "Turn  handle 
to  left,  remove  RT  Unit.  Insert  replacement  RT  Unit,  turn  handle  to  right." 
In  the  personalized  style,  the  Instruction  would  say,  "Now,  turn  the  handle  to 
the  left  and  pull  the  RT  out.  OK,  now  you  are  ready  to  put  the  new  RT  in. 
After  it  Is  in  place,  turn  the  handle  to  the  right." 

Technical  Data  Format  Design.  The  Fully  Procedural ized  Job  Performance 
Aid  (FPJPA)  approach  developed  by  AFHRL  (Joyce,  Chenzoff,  Mulligan,  &  Mallory, 
1973)  was  used  as  the  starting  point  In  developing  formats  for  presentation  of 
data  on  a  computer-based  system.  Analysis  of  the  FPJPA  format  revealed 
several  features  which  were  recommended  for  Incorporation  in  the  automated 
system.  The  principal  features  proposed  for  retention  were: 

1.  Dual -Level  Presentation.  The  FPJPA  dual-level  feature  presents 
information  in  both  boldface  and  regular  print.  The  regular  print  provides 
detailed  instructions  on  how  to  do  the  task  and  is  intended  for  technicians 
with  limited  experience  on  the  task.  The  boldface  print  outlines  the  major 
steps  for  completing  the  task  and  is  Intended  for  the  experienced  technician 
who  is  fully  qualified  on  the  task  and  does  not  require  the  detailed  Instruc¬ 
tions.  The  authors  recommended  that  the  dual -level  concept  be  applied  and 
extended  to  provide  three  or  more  levels  of  detail  or  "tracks"  depending  upon 
the  complexity  of  the  task.  They  noted  that  an  automated  system  can  make  much 
better  use  of  the  concept  since  it  can  support  any  number  of  levels  of  detail. 
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A  three-track  approach  was  developed.  The  tracks  would  provide  data  at 
three  levels  of  detail  appropriate  for  experienced,  average,  and  inexperienced 
technicians.  It  was  suggested  that  three  levels  of  detail  are  adequate  for 
most  tasks  but  that  four  or  even  five  levels  may  be  required  for  special 
circumstances.  Examples  of  instruction  frames  presented  in  the  three  basic 
tracks  are  presented  in  Figures  1,  2,  and  3.  The  Track  1  instructions  (Figure 
1)  provide  only  critical  information  as  reminders  of  critical  steps,  warnings, 
and  cautions.  These  instructions  are  provided  for  highly  experienced  techni¬ 
cians  who  have  performed  the  task  many  times.  The  Track  2  instructions 
(Figure  2)  present  information  in  the  form  of  a  checklist,  with  supporting 
graphics  on  call.  These  instructions  are  for  technicians  who  have  performed 
the  task  many  times  but  are  not  yet  "well  experienced."  The  Track  3  instruc¬ 
tions  (Figure  3)  provide  detailed  procedures  supported  by  abundant  graphics 
information.  These  instructions  are  intended  for  the  inexperienced  technician. 

2.  Use  of  Line  Drawings.  FPJPAs  provide  line  drawings  keyed  to  the 
instructions  to  help  the  technician  locate  referenced  parts.  Similar  drawings 
were  recommended  for  the  automated  system.  It  was  pointed  out  that  FPJPA 
drawings  ?  usually  very  detailed,  but  that  less-detailed  drawings  would  be 
required  automated  presentation  (to  reduce  storage  requirements). 

3.  ation  Instructions.  FPJPAs  provide  a  summary  of  information 
jplies,  warnings,  cautions,  etc.)  required  to  perform  a  task  at  the 

j  of  a  procedure.  This  feature  is  similar  to  the  Input  Conditions 
£  ..  ,  of  a  standard  Job  Guide  Manual.  The  authors  proposed  that,  with  an 
automated  system,  the  information  could  be  presented  at  points  in  the 
procedure  where  it  would  be  the  most  beneficial. 
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Figure  1.  Example  of  Track  1  Frame  for  Remove  and  Replace  Task 


i«t«u  tic  mm  r«ma  out  **t« 


! 

i 


1M  FIRST  TA9I  IS  TO  I 
CONTROL  TALC  FROM  TIC 
HOOKING  ON. 


TIC  STARTER 

:  tow  mi 


I'LL  MM  TOU  A  SERIES  Or  STEPS.  OC  AT  A 
TINE •  MEN  TOU  F INI 91  A  STEP.  LET  ME 

knom.  and  i'll  present  tic  ic*t  step 
description. 


TIC  FIRST  STEP  IN  THIS  TA9*  IS  TO: 

0E-EMER6I2E  TiC  STARTER  ELECTRICAL  CIRCUITS 
rOR  TiC  ENGINE  TOU'RE  MOWING  ON 


////////I//////////////// 
AVAILABLE  REVS:  OuTLlIC  TASK. 

STEP  COPLETE 
///////////////////////// 


Figure  2. 

I 


Example  of  Track  2  Frame  for  Remove  and  Replace  Task. 
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Figure  3.  Example  of  Track  3  Frame  for  Remove  and  Replace  Task 


4.  Formats  for  Troubleshooting,  For  troubleshooting,  the  authors 
proposed  using  the  FPJPA  format  for  inexperienced  technicians,  a  logic  tree 
format  for  average-experience-level  technicians,  and  a  deductive  trouble¬ 
shooting  aid  format  for  highly  experienced  technicians.  The  deductive 
troubleshooting  aid  would  provide  only  basic  information  needed  to  trouble¬ 
shoot.  The  basic  information  would  include  schematics,  wiring  diagrams,  and 
similar  types  of  information. 

Pool  Concept.  The  use  of  pools  of  support  information  was  also  proposed. 
Pool  information  is  defined  as  back-up  information  which  may  be  made  available 
at  a  given  point  in  the  procedure  where  it  may  be  needed  by  some  users.  The 
pool  concept  is  described  as  tailoring  information  to  meet  an  individual's 
specific  information  needs.  It  provides  a  means  for  providing  supplemental  or 
remedial  instructions  for  portions  of  a  task  that  are  unfamiliar.  For 
example,  if  a  technician  is  directed  within  a  procedure  to  calibrate  an 
oscilloscope  but  does  not  remember  how,  he  would  be  able  to  call  up  a 
procedure  from  the  "pool"  on  how  to  calibrate  the  scope.  Several  types  of 
information  were  suggested  as  pool  information. 

1.  Significance  Information.  Provides  an  explanation  of  the  significance 
of  the  work  being  done  in  terms  of  flight  safety  and  functional  integrity. 

2.  Overview  of  Maintenance  Task.  Provides  a  preview  summary  of  task  to 
be  performed. 

3.  Part,  Supply,  and  Tool  Descriptions.  Provide  information  on  parts, 
supplies,  and  tools  including  cross-references  of  serial  and  identification 
numbers.  Graphics  may  be  provided. 

4.  Use  of  Tools  and  Test  Equipment.  Provides  reference  information  on 
the  correct  use  of  tools  and  test  equipment. 

5.  Subsystem  Functional  Descriptions.  Provide  information  on  functions 
and  operation  of  subsystems. 

6.  Diagrams  and  Schematics.  Provide  the  appropriate  diagrams  and 

schematics  for  the  specified  equipment. 

7.  Preference  Performance  Descriptions.  Provide  information  such  as 
special  terminology  and  vocabulary,  basic  maintenance  procedures,  etc. 

Graphics  Simplification.  Examination  of  typical  illustrations  used  in 
technical  orders  revealed  that  the  illustrations  are  highly  detailed.  Due  to 
the  large  amount  of  computer  memory  required  to  store  detailed  graphics, 
techniques  are  needed  to  reduce  the  detail  and  complexity  of -the  graphics.  An 
analysis  of  illustration  requirements  was  made  to  identify  methods  to  simplify 
graphics. 

The  analysis  viewed  graphics  as  combinations  of  discriminative  and 
nondiscrimi native  stimuli.  Discriminative  stimuli  provide  a  cue  to  the 
identification  of  an  object.  Nondiscrimi native  stimuli  do  not  provide  a  cue 
to  the  identification  of  the  object  In  question  (although  they  might  provide  a 
cue  to  the  identification  of  other  objects  on  the  graphic).  Discriminative 


stimuli  provide  useful  information.  Nondiscrimi native  cues  provide  no  useful 
Information  for  the  task  at  hand.  Therefore,  graphics  provided  for  the 
purpose  of  aiding  the  user  in  identifying  components  referenced  In  the  text 
could  be  simplified  by  including  only  the  discriminating  stimuli  and  leaving 
out  the  nondiscriminating  stimuli.  Thus,  a  graphic  illustrating  a  piece  of 
equipment  need  include  only  enough  detail  (discriminating  stimuli)  to  locate 
the  item  of  concern.  Other  detail  (nondiscriminating  stimuli)  can  be 
eliminated  from  the  graphic,  saving  both  storage  space  and  the  time  required 
to  draw  the  detail  on  the  screen. 

Figure  4  demonstrates  the  application  of  this  approach  to  graphics 
simplification.  Graphic  A  is  a  drawing  of  an  oscilloscope  from  the  angle  at 
which  the  user  would  view  it.  This  graphic  shows  all  of  the  key  elements  on 
the  face  of  the  oscilloscope,  but  not  with  the  detail  shown  in  the  technical 
order.  Graphic  6  shows  a  simplified  Illustration  which  might  be  used  in 
support  of  an  Instruction  to  turn  on  the  scope.  It  includes  only  the  key 
geographical  cues  to  show  the  approximate  location  on  the  scope  and  enough 
surrounding  detail  to  pinpoint  the  specific  component.  Graphic  C  shows  how  a 
subsequent  graphic  might  be  used  to  locate  another  component  of  the  same 
scope.  Graphics  B  and  C  illustrate  the  fact  that  a  stimuli  may  be  a 
discriminating  cue  in  one  case  and  a  nondiscriminating  cue  in  another  case. 
There  appears  to  be  no  particular  justification  for  presenting  full  detail  of 
an  equipment/assembly  needed  only  for  locator  purposes. 


Figure  4.  Example  of  Graphics  Simplification  Concept. 

Frazier,  Campbell,  and  Kniess  (1979)  noted  that  the  use  of  computer 
graphics  for  presenting  an  Illustrated  parts  breakdown  (IPB)  presents  a 
special  problem.  The  highly  complex  drawings  require  large  amounts  of  core 
memory  and  require  displays  with  very  fast  drawing  speeds  to  prevent  flicker. 
The  precision  and  resolution  of  the  display  become  Important  when  displaying 
IPB  graphics.  For  this  reason,  simplification  of  IPB  graphics  is  essential. 
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Fine  detail  is  more  important  in  IPB  graphics  since  small  details  may  be 
important  in  discriminating  small  parts.  An  automated  system  should  give  the 
technician  the  capability  to  discriminate  fine  detail  from  several  feet  away. 
Zooming  the  illustration  can  be  used  to  achieve  maximum  viewability.  However, 
this  may  cause  other  desired  detail  to  move  out  of  the  viewing  area.  For  this 
reason,  when  the  zoom  feature  is  used,  the  technician  must  be  able  to  control 
the  degree  of  zoom  to  achieve  the  best  combination  of  detail  and  viewing  area. 
A  joystick  is  suggested  as  a  means  of  providing  control. 

Figure  5  illustrates  an  approach  proposed  for  organizing  a  large  complex 
graphic  for  computer  display.  For  a  complex  assembly,  the  entire  assembly  is 
shown  in  limited  detail.  The  user  then  has  the  option  to  select  designated 
sections  of  the  assembly  for  detailed  viewing.  This  approach  can  provide  a 
solution  to  the  overcrowding  of  the  graphics  display  while  still  providing  the 
required  detail  without  using  the  zoom  technique.  This  approach  is  shown  in 
Figures  6  and  7.  Figure  6  shows  a  limited-detail  view  of  the  whole  assembly. 
Figure  7  demonstrates  how  a  more  detailed  view  of  a  section  of  the  overall 
assembly  can  be  displayed. 


Figure  5.  Model  for  Graphics  Simplification 


Figure  7.  Graphic  Detail  of  Subsection  1 
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The  presentation  of  schematics  and  block  diagrams  presents  similar 
problems.  Typical  blocked  schematics  and  wiring  diagrams  presented  in 
technical  orders  are  too  large  for  direct  transliteration  and  presentation  on 
a  cathode-ray  tube  (CRT)  Mithout  an  unacceptable  loss  of  detail.  Thus,  it  Mas 
necessary  to  develop  a  simplified  means  of  breaking  the  diagrams  into  smaller 
diagrams  Mhich  are  suitable  for  presentation  on  a  CRT  Mhile  providing  the 
technician  Mith  a  means  of  maintaining  his  orientation  Mithin  the  total  realm 
(cognitive  field)  represented  by  the  schematic.  Several  potential  approaches 
Mere  considered. 

One  approach  proposed  for  screen  presentation  of  large,  complex  draMings 
Mas  to  shoM  a  simplified  overall  diagram  illustrating  the  energy  floM 
relationships  Mith  numeric  identifiers  Mhich  alloM  the  technician  to  call  up  a 
more  detailed  presentation  of  the  subsection  of  the  diagram.  Figure  8 
presents  an  example  of  hOM  the  overall  diagram  could  be  presented.  In  this 
diagram,  the  components  and  interconnections  are  shoMn  Mithout  Identifying 
Information.  Figure  9  presents  an  example  of  a  frame  that  Mould  be  presented 
in  response  to  a  request  to  see  subsection  5  of  the  diagram  in  detail. 

Another  proposed  approach  Is  the  use  of  "matchlng-to-sample"  techniques 
for  comparisons  betMeen  functional  diagrams  or  blocked  schematics  and 
electrical  schematics.  Figure  10  presents  an  example  of  this  approach.  The 
functional  description  for  the  board  is  shoMn  In  the  top  of  the  Illustration. 
It  is  matched  Mith  the  blocked  schematic  shoMn  in  the  lOMer  section  of  the 
Illustration.  The  functional  description  is  ordered  and  positioned  to  match 
the  blocked  schematic  beloM. 
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Figure  8.  Simplified  Overall  Miring  Diagram. 


19 


Graphics  Cuing  Techniques.  Several  graphics  cuing  techniques  are  avail- 
able  which  enhance  the  presentation  of  technical  data.  Numeric  and  labeling 
cues  are  essential  for  relating  Illustrations  to  textual  material  and 
providing  a  means  of  retrieving  pool  data.  Numeric  cues  have  two  main  uses. 
They  can  be  used  following  a  component  name  to  identify  the  component  on  a 
corresponding  Illustration.  They  can  also  be  used  to  request  that  the 
computer  provide  specified  information  on  a  given  component  shown  on  an 
illustration. 

Line  parameters  can  also  be  used  as  graphics  cues.  Various  line  param¬ 
eters  should  be  adjustable  for  use  as  cues.  Several  degrees  of  line  thickness 
can  be  used  in  an  illustration  to  communicate  specific  Information.  For 
example,  an  extra-thick  line  could  be  used  to  Indicate  the  major  energy  flow 
paths  in  a  blocked  schematic.  Another  line  parameter  which  can  be  varied  Is 
degree  of  brightness.  Level  of  brightness  could  be  used  to  Indicate 
foreground/background  relationships  or  to  highlight  a  signal  flow  of  Interest 
on  a  schematic.  Dashed  lines  could  be  used  to  delineate  sections  of  an 
Illustration.  For  example,  dashed  lines  could  be  used  to  separate  functional 
sections  of  a  schematic. 

Color,  shading,  cross-hatching,  and  blinking  are  additional  cuing 
techniques  with  potential  for  use  In  automated  technical  data  systems.  These 
cues  were  not  Investigated  in  the  studies.  However,  It  was  recommended  that 
they  be  studied  to  determine  If  they  enhance  the  discrimination  of  the 
technician. 

Pseudoanimation.  Pseudoanimation  refers  to  a  technique  for  presenting 
procedural  Instructions  without  using  textual  materials.  A  series  of  static 
drawings  (line  drawings)  are  used  for  Illustrating  how  manual  tasks  should  be 
performed.  This  approach  Is  based  upon  the  operant  conditioning  concept  of 
behavioral  chaining  In  which 

...the  performance  of  an  act  Is  viewed  as  a  serial  chain  of 
behavioral  operations.  Each  member  leads  to  the  next  until  the 
occurrence  of  the  terminal  member  that  leads  to  reinforcement.  The 
potential  applicability  of  pseudoanimation,  therefore.  Is  that  of 
providing  discriminative  cues  that  guide  Imitative  responses,  the 
execution  of  which  constitutes  a  recommended  manually  performed 
procedure.  (Frailer  et  al.,  1979,  p.  46) 

Since  pseudoanimation  provides  Instructions  on  how  to  do  a  task  without  the 
use  of  verbal  descriptions,  it  can  be  said  to  be  "culture-  or  language-free." 
Pseudoanimation  was  only  partially  evaluated  In  these  studies. 

The  Illustrations  used  In  pseudoanimation  are  digitized  drawings  showing 
technicians  performing  the  task.  The  line  drawings  are  simplified  by 
eliminating  unnecessary  detail.  Two  approaches  may  be  used  in  developing 
pseudoanimation  Instructions.  The  first  approach  uses  one  Illustration  to 
depict  a  procedural  step.  The  second  approach  uses  two  illustrations  on  the 
same  frame  to  depict  each  step.  The  first  Illustration  presents  the  Initial 
condition  (e.g.»  position  of  hand)  and  the  second  Illustration  presents  the 
condition  after  the  step  has  been  completed.  Figure  11  presents  an  example  of 
the  second  approach.  The  Illustration  on  the  left  presents  the  Initial  hand 


position.  The  Illustration  on  the  right  presents  the  terminal  hand  position 
when  the  step  has  been  completed. 


Figure  11.  Example  of  Pseudoanlmatlon  Frame  Using  Two  Illustrations. 

Pseudoanimation  was  not  fully  evaluated  In  these  studies.  The  authors 
believed  that  it  had  potential  for  use  primarily  with  personnel  with  limited 
language  skills  or  in  cases  where  a  given  set  of  data  Is  to  be  used  by 
personnel  of  several  nationalities  and  translation  Into  several  languages 
would  not  be  practical.  Further  study  Is  recommended.  Of  particular  concern 
is  determining  the  maximum  degree  of  complexity  of  tasks  which  can  be 
presented  effectively  using  pseudoanimation. 


Phase  II.  Preliminary  laboratory  Demonstrations 

After  the  human  factors  requirements  analysis  was  completed,  the  next  step 
was  to  develop  a  preliminary  laboratory  prototype  of  a  computer-based 

maintenance  aids  system  to  test  and  demonstrate  the  concepts  developed.  The 
system  developed  was  not  intended  to  be  representative  of  a  computer-based 
maintenance  aids  system  but  to  provide  a  research  tool  for  testing  and 

refining  the  concepts  developed  on  a  preliminary  basis.  The  preliminary 
system  included  the  basic  elements  required  for  a  computer-based  maintenance 
aid:  a  minicomputer,  a  graphics  terminal,  a  floppy  disk  drive,  and  associated 
peripherals.  The  components  are  briefly  described  below. 

A  16-bit  minicomputer  with  64K  bytes  of  main  memory  served  as  the  host 
unit.  It  was  Interfaced  to  a  Hughes  CRT  graphics  terminal.  The  terminal  had 
a  9  x  12  inch  viewing  area,  with  a  viewable  matrix  of  1536  x  2048  pixels.  It 

had  a  6-to-l  200m  capability.  The  terminal  had  a  continuous  zoom  capability 


for  positioning  and  enlarging  the  display  image.  The  zoom  and  image 
positioning  were  controlled  by  a  joystick.  The  writing  speed  was  1.760  inches 
per  second  for  long  and  short  vectors.  The  writing  speed  for  characters  was 
2,000  per  second.  The  system  had  linear  and  conic  vector-generation 
capabilities  as  a  standard  hardware  feature.  Also,  it  was  capable  of  2-0 
rotation  and  translation.  The  terminal's  keyboard  was  relabeled  and  redefined 
via  software  to  provide  a  means  of  simulating  the  function  key  concept. 

External  memory  was  provided  by  a  dual  floppy  disk  system.  A  digitizing 
table  and  controller  were  used  to  digitize  graphic  materials.  A  high-speed 
printer  and  carousel  terminal  were  also  available  for  programming  purposes. 

Technical  data  for  three  tasks  were  developed  for  use  In  a  preliminary 
evaluation  and  demonstration  of  the  concepts.  The  three  tasks  were  a  remove 
and  replace  task,  a  troubleshooting  task,  and  a  trainlng/faml liarization 
task.  The  remove  and  replace  task  was  adapted  from  a  Job  Guide  Manual  for  the 
C-141  aircraft.  It  was  used  to  demonstrate  the  three-track  and  pool  concepts. 
Extensive  opportunities  to  switch  back  and  forth  between  tracks  and  to  call  up 
pool  data  were  provided.  It  featured  simplified  graphics,  supervisor/ 
technician  discourse,  and  a  simulated  function  keyboard. 

The  troubleshooting  procedure  was  adapted  from  a  portion  of  a  fully 
procedural i zed  troubleshooting  aid  for  a  navigational  computer  (AN/ASN-35). 
This  procedure  was  used  to  demonstrate  the  efficient  computer  management  of 
the  fault-tree  troubleshooting  approach.  It  also  demonstrated  the  placement 
of  tolerance  data  at  the  point  required  and  the  ready  access  to  reference 
material  to  minimize  search  time. 

The  third  procedure  was  used  to  demonstrate  the  potential  use  of  the 
system  as  an  aid  for  training  and  equipment  familiarization.  The  procedure 
provided  Instructions  for  the  setup  and  calibration  of  the  Textronlx  535B 
oscilloscope.  The  goal  was  to  show:  (a)  how  a  newly  assigned  technician  can 
use  the  system  for  rapid  familiarization  with  special  or  new  test  equipment, 
and  (b)  how  an  experienced  technician  can  use  the  system  for  refresher 
training  on  equipment  that  he  has  not  used  recently.  The  procedure  also 
provided  an  example  of  how  supplemental  procedures  could  be  used  as  pool 
data.  For  example,  if  a  technician  were  Instructed  to  use  the  oscilloscope 
but  could  not  remember  how  to  set  It  up,  he  could  retrieve  the  procedure  from 
the  pool. 

The  demonstrations  were  successful  In  Illustrating  the  concepts  developed 
in  the  human  factors  analysis  and  proving  the  overall  feasibility  of  a 
computer-based  maintenance  aids  system.  In  addition,  they  provided  the  basis 
for  refinement  of  the  overall  system  and  the  concepts  developed.  Specific 
refinements  were  Incorporated  In  the  system  in  preparation  for  a  more  complete 
demonstration  in  the  next  phase. 


Phase  HI.  Preliminary  System  Evaluation 

The  objective  of  this  phase  was  to  conduct  a  more  thorough  evaluation  of 
the  concepts  and  the  prototype  system.  This  was  accomplished  by  developing 
data  in  the  proper  formats  for  two  maintenance  tasks  and  having  experienced 
maintenance  technicians  use  the  system  to  perform  simulated  maintenance  tasks. 
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Sample  Technical  Oata  Development.  Technical  data  were  developed  for  two 
tasks  for  use  \n  the  evaluation.  They  were  a  remove  and  replace  task  and  a 
troubleshooting  task. 

1.  Remove  and  Replace  Task.  The  remove  and  replace  task  was  the  same 
task  (remove  and  replace  engine  starter  control  valve)  as  that  used  for  the 
earlier  demonstrations.  For  this  evaluation,  the  data  were  extended  and  put 
Into  a  fully  procedural  1  zed  job  performance  aid  format.  The  data  were 
presented  In  three  levels  of  detail  or  tracks.  The  data  are  described  In  the 
following  paragraphs. 

Track  3  data  were  designed  for  the  Inexperienced  technician  or  for 
experienced  technicians  with  limited  knowledge  of  the  task.  This  track  was 
fully  procedural  1  zed  with  relatively  fine  step  sizes.  The  procedures  were 
keyed  to  locator  Illustrations  to  facilitate  location  of  referenced 
components.  Extensive  pool  Information  was  made  available.  Figure  3  Is  an 
example  of  a  frame  presented  at  the  Track  3  level. 

Track  2  data  were  designed  for  technicians  who  are  familiar  with  the  task 
but  have  not  mastered  It.  This  track  provided  procedural  Information  using 
relatively  coarse  procedural  steps  (turn  off  power— without  explanation  of 
how).  Locator  Illustrations  were  not  usually  available  unless  the  technician 
requested  them.  Pool  data  were  available  for  reference  when  needed.  Figure  2 
Is  an  example  of  a  frame  presented  at  the  Track  2  level. 

Track  1  data  were  designed  for  7-skl  11  -level  and  some  5-skl  11 -level 
technicians  with  relatively  extensive  experience  on  the  task.  This  track  was 
even  less  detailed.  It  consisted  primarily  of  general  and  special  repair 
notes,  quantitative  information  (tolerances,  capacities,  etc.),  and  warnings 
and  cautions.  Notes  were  expressed  In  procedurallzed  fashion.  The  list 
formats  key  was  available  In  case  the  technician  found  that  he  needed  more 
Information.  Figure  1  Is  an  example  of  a  frame  presented  at  the  Track  1  level. 

After  the  remove  and  replace  task  was  completed,  the  display  shown  to  the 
technician  moved  directly  to  a  checkout  procedure  and  then  to  the  proper  data 
for  any  required  follow-on  tasks.  All  text  was  written  In  a  conversational 

mode.  An  available-keys  box  was  provided  at  the  bottom  of  each  frame.  This 

was  used  to  list  the  function  keys  that  were  active  at  that  time.  Pressing 
any  other  key  resulted  In  an  error  message. 

2.  Troubleshooting  Task.  Two  types  of  troubleshooting  aids  were 

developed!  a  fully  procedural  1  zed  troubleshooting  aid  and  a  deductive 
troubleshooting  aid.  The  fully  procedurallzed  troubleshooting  aid  was 
designed  to  troubleshoot  the  AN/ASN-35  navigational  computer  to  Identify  a 
faulty  circuit  card  and  to  Isolate  a  fault  on  that  card  (Schmitt  Trigger 

Board).  The  deductive  aids  were  designed  to  troubleshoot  the  card  to  Identify 
a  faulty  component  on  the  card.  The  fully  procedurallzed  troubleshooting  aids 
were  similar  In  format  and  level  of  detail  to  those  used  for  the  remove  and 
replace  task  and  are  not  further  described  here. 

The  deductive  troubleshooting  aid  made  four  types  of  reference  data 
available  to  the  technician  and  allowed  him  to  create  his  own  troubleshooting 
strategy.  The  technician  was  given  a  choice  of:  (a)  blocked  schematic 
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diagrams;  (b)  a  functional  description;  (c)  electrical  schematic;  and  (d) 
parts  locator  diagram.  The  technician  was  able  to  select  any  of  the  above 
options  and  to  switch  back  and  forth  between  them  as  he  wished.  In  addition, 
a  variety  of  pool  Information,  such  as  theory  of  operation  or  procedural 1  zed 
instructions  on  how  to  make  various  checks,  was  available. 

3.  Illustrated  Parts  Breakdown  (IPB).  A  limited  sample  of  IPB  data  were 
developed  for  evaluation.  The  data  used  the  simplified  overall  assembly 
approach  described  above  under  Phase  I. 

Evaluation  Procedure.  Sixteen  experienced  Air  Force  Reserve  technicians 
served  as  subjects  for  the  evaluation.  Each  technician  completed  both  tests 
under  simulated  conditions. 

For  the  remove  and  replace  task,  there  was  no  equipment  available  to  allow 
the  technicians  to  actually  perform  the  task  "hands-on."  Therefore,  the 
technicians  simply  viewed  the  data  on  the  display  and  interacted  with  the 
system.  After  completing  each  task,  the  technicians  were  asked  to  complete  a 
questionnaire  regarding  format,  display  options,  and  other  features  of  the 
system. 

For  the  troubleshooting  task,  each  technician  used  the  fully  proce¬ 
dural  ized  troubleshooting  aid  to  troubleshoot  the  computer  to  identify  the 
faulty  card.  At  that  point,  he  was  given  the  choice  of  troubleshooting  the 
card  using  the  fully  procedurallzed  aid  or  using  the  deductive  aid.  The 
AN/ASN-35  navigational  computer  was  available  for  the  test.  However,  the 
necessary  test  equipment  was  not  available  for  troubleshooting  the  system  to 
the  card  level.  To  realistically  simulate  troubleshooting  the  computer,  each 
technician  went  through  the  procedure  using  the  computer  (flipping  switches, 
locating  test  points,  etc.).  When  a  test  was  called  for,  the  experimenter 
provided  the  technician  with  the  results  of  that  test.  Since  troubleshooting 
the  Schmitt  Trigger  Board  required  only  a  voltmeter  and  oscilloscope,  it  was 
possible  for  the  technicians  to  actually  perform  this  task.  Each  technician 
completed  a  questionnaire  after  completing  the  troubleshooting  task. 

The  IPB  data  were  evaluated  by  having  the  technicians  manipulate  the  data 
base  and  then  having  them  complete  a  questionnaire  to  give  their  reaction. 

Results.  The  results  of  the  questionnaires  are  presented  in  Tables  2  and 
3.  In  addition  to  the  scaled  questions  reported  in  the  tables,  technicians 
were  asked  to  give  verbal  comments  on  what  they  liked  and  disliked  and  to  give 
suggestions  for  improving  the  system.  Representative  observations  drawn  from 
their  comments  are  summarized  below: 

1.  The  technicians'  comments  on  the  system  were  generally  positive. 
Positive  feelings  were  expressed  in  comments  such  as  "it  is  a  powerful 
training  tool,"  "good  guide  to  completing  routine  tasks,"  "flexible  in 
allowing  the  technician  to  work  at  his  own  level,"  "impressive,"  "anyone  off 
the  street  would  be  able  to  do  it,"  and  "liked  the  detail  level  of  the 
instructions." 

2.  The  tracks  and  pools  were  described  as  the  most  useful  features  of  the 
system. 


i 

j  3.  The  graphics  were  considered  to  be  the  least  useful  feature.  They 

i  were  described  by  several  technicians  as  "ambiguous,"  "sloppy,"  and  "unclear." 


4.  The  system  should  be  made  capable  of  presenting  schematic  information 
simultaneously  with  component  layout  information. 


5.  The  schematics  should  be  labeled  to  facilitate  comparison  with 
component  layout  diagrams. 


Table  2.  Summary  of  Questionnaire  Results  on  FPJPA  Format  Design* 


1 


) 

1 


) 


f 


Issue 

Finding 

Number  of 
participants 

1.  Overall 

Effectiveness 

Excellent 

7 

Good 

8 

Fair 

1 

2.  Affective  Reactions 

a.  Working  with  System 

Very  enjoyable 

7 

Pleasant 

7 

Neutral 

2 

b.  Language  Used 

Informal  and  relaxed 

7 

Formal  and  rigid 

1 

Between 

8 

c.  Dialog 

Enjoyable  and  efficient 

6 

Efficient 

7 

Enjoyable 

3 

3.  Completeness 

Complete  enough 

15 

Not  complete  enough 

1 

4.  Comprehension 

Easy  to  understand 

12 

Fairly  easy  to  understand 

4 

5.  Dialog  Usability 

Collection  of  facts 

10 

Complete  facts 

3 

Like  a  person 

3 

6.  Ability  to  Work 
at  One's  Level 

tssily  possible 

14 

Possible  with  effort 

1 

Impossible 

1 

7.  Function  Key 

Covered  everything  needed 

12 

Completeness 

Additional  suggestions 

2 

*Adapted  from  Frazier  et  al.,  1979,  p.  54. 
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6.  Having  to  switch  back  and  forth  between  diagrams  and  pool  information 
was  consistently  reported  as  the  least  useful  feature  of  the  deductive  trouble¬ 
shooting  aid.  Simultaneous  display  of  several  types  of  information  was 
suggested  as  the  solution  to  this  problem. 

7.  Inability  to  present  full  labeling  of  complex  system  schematics  was 
seen  as  reducing  the  usability  of  schematic  information. 

8.  The  use  of  system  guidance  which  provided  the  technicians  with  the 
benefits  of  the  knowledge  of  experienced  technicians  was  seen  as  the  most 
useful  feature  of  the  fully  proceduralized  troubleshooting  aids. 

Table  3.  Summary  of  Questionnaire  Results  on  Deductive  Aids3 

Number  of 

Issue  _ Finding _ participants 


1.  Overall  Effectiveness 

Excellent 

8 

Good 

3 

2.  Affective  Reactions 

a.  Working  with  System 

Very  Enjoyable 

5 

Pleasant 

7 

Neither 

4 

b.  Language  Used 

Informal  and  Relaxed 

5 

Formal  and  Rigid 

1 

Between 

10 

c.  Dialog 

Enjoyable  and  Efficient 

6 

Enjoyable 

3 

Efficient 

4 

3.  Completeness 

Complete  Enough 

16 

4.  Comprehension 

Easy  to  Understand 

11 

Fairly  Easy  to  Understand 

4 

5.  Dialog  Usability 

Collection  of  Facts 

11 

Complete  Facts 

2 

6.  Ability  to  Work 

Easily  Possible 

11 

at  One's  Level 

Possible  with  Effort 

2 

Impossible 

2 

7.  Function  Key 

Covered  Everything 

13 

Completeness 

Additional  Suggestions 

2 

aAdapted  from  Frazier  et  al.,  1979,  p.  56. 


9.  The  presentation  of  textual  information  on  the  same  screen  as  the 
graphic  aids  was  seen  as  the  most  useful  feature  of  the  deductive  trouble¬ 
shooting  aids. 


10.  Technicians  responded  favorably  to  the  provision  of  estimates  of 
fault  probabilities  based  on  other  technicians'  experience  in  troubleshooting 
the  same  system. 

11.  The  printed  circuit  layout  diagrams  were  seen  as  the  most  useful  of 
the  deductive  aids  provided.  Theory  of  operation  and  other  information 
supporting  improved  "in-the-head"  knowledge  were  seen  as  useful  but  secondary. 

12.  The  technicians'  evaluations  of  the  IPB  data  were  uniformly  positive. 
The  zoom  capability  was  often  cited  as  a  strong  positive  factor. 

13.  It  was  suggested  that  a  reduced-size  overall  view  of  the  assembly  be 
provided  in  the  corner  when  a  detailed  illustration  of  a  section  of  the 
assembly  is  presented. 


Phase  IV.  Synthesis  of  Results 

In  this  final  phase  of  the  effort,  the  findings  were  compiled  and  the 
fundamental  requirements  for  a  computer-based  maintenance  aids  system  were 
developed.  The  fundamental  system  requirements  were  based  on  criteria 
developed  in  the  studies.  The  criteria  were  developed  "solely  with  the  needs 
of  the  maintenance  technician  in  mind"  (Frazier  et  al.,  1979,  p.  65).  The 
authors  emphasized  that  "...any  candidate  system  intending  to  disseminate 
information  to  maintenance  technicians  must  be  able  to  meet  these  requirements 
if  the  system  is  to  be  considered  usable"  (Frazier,  op.  ci t. ) .  The  basic 
requirements  for  a  computer-based  maintenance  aids  system  are  described  below. 

Basic  System  Design  Requirements.  The  system  must  meet  the  following 
requirements. 

1.  Be  truly  interactive  and  communicate  with  the  technician  in  natural 
and  acceptable  ways. 

2.  Use  a  function  key  panel  that  will  allow  the  technician  to  retrieve 
information  in  an  easy  and  convenient  manner. 

3.  Support  the  use  of  improved  job  performance  aids,  including 
procedural i zed,  nonprocedural! zed,  deductive,  and  directive  aids. 

4.  Use  multiple  levels  of  detail  (tracks)  to  provide  information 
tailored  to  the  experience  and  skill  level  of  the  user. 

5.  Use  information  "pools"  to  provide  reference  information  and  "fill  in 
gaps"  in  the  individual's  knowledge. 

6.  Use  simplified  graphic  techniques  which  eliminate  unnecessary  detail. 

7.  Be  fully  portable,  ruggedized,  and  suitable  for  use  in  a  variety  of 
environmental  conditions. 

8.  Provide  reference  data  when  and  where  needed. 

9.  Provide  for  quick  retrieval  and  display  of  data. 


10.  Collect  performance  data  for  training,  evaluation,  supervisory,  and 
logistics  management  purposes. 

Hardware  Requirements.  The  major  hardware  requirements  are  summarized 
below. 

1.  Portability.  A  system  for  use  on  the  flightline  must  be  suitable  for 
transportation  By  two  people.  It  should  be  easily  disassembled  and  re¬ 
assembled.  The  weight  of  any  component  should  not  exceed  approximately  60 
pounds.  The  system  should  resist  the  normal  shock,  vibration,  humidity,  and 
heat  that  may  be  expected  in  the  maintenance  work  environment.  The  keyboard 
should  be  able  to  survive  a  4-  to  5-foot  drop  onto  a  hard  surface.  It  is 
anticipated  that  the  system  will  be  mounted  on  a  cart. 

2.  Host  Computer.  The  host  computer  should  be  a  mini-  or  microcomputer 
with  a  minimum  of  64K  bytes  of  main  memory.  This  recommendation  assumes  that 
a  separate  graphics  processor  (graphics  terminal)  with  32K  bytes  of  core 
memory  will  be  provided  in  a  graphics  terminal.  If  a  graphics  processor  is 
not  provided,  an  additional  32K  bytes  of  main  memory  will  be  required  for  the 
host  computer. 

3.  Graphics  Terminal.  A  separate  graphics  processor  with  32K  bytes  of 

main  memory  is  required.  The  terminal  (or  the  terminal  and  the  host  computer) 
must  provide  the  following  capabilities: 

a.  Scaling  and  Positioning.  The  system  must  have  a  capability  to 
zoom  an  imaged  A  6-to-l  zoom  capability  is  recommended.  The  system  must  also 
have  the  capability  to  reposition  the  image  in  conjunction  with  large 
graphics. 

b.  Function  Generators.  Character  generators  for  all  standard 

characters,  with  adjustable  sizes,  are  required.  Conic  and  linear  function 

generators  are  required  for  data  compression  purposes. 

c.  Graphics  Display.  The  graphics  display  should  have  a  resolution 
of  no  less  than  120  pixels  per  inch.  The  viewing  area  should  be  at  least  108 
square  inches. 

d.  Writing  Speed.  The  writing  speed  should  be  no  less  than  2,000 
inches  per  second  for  graphics. 

4.  Direct  Access  Storage.  The  prototype  system  should  have  at  least  15 
megabytes  of  direct  access  storage.  This  amount  assumes  that  the  system  is 
loaded  from  a  larger  archival  data  base. 

5.  Keyboard.  A  special  function  keyboard  should  be  provided.  The 

keyboard  should  provide:  the  function  keys  described  earlier,  standard 

alphabetic  characters,  a  numeric  keypad,  and  controls  for  scaling  and 
positioning  the  graphic  data.  The  keyboard  should  have  a  backlighting 

capability  so  that  those  keys  currently  active  may  be  lit. 

6.  Power.  The  system  should  operate  on  400-cycle,  110-volt  power  and 
should  Incorporate  switcher-type  power  supplies. 
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Software  Requirements.  The  major  software  requirements  are  summarized 
below. 

1.  Computer  Language.  The  system  should  provide  a  Fortran  compiler. 
This  requirement  is  based  on  the  fact  that  most  commercially  available 
software  packages  for  graphics  are  written  in  Fortran. 

2.  Graphics  Software.  A  graphics  software  library  should  be  provided 
with  capabilities  for: 

a.  Positioning  the  cursor; 

b.  Drawing  figures  such  as  curves,  arcs,  circles,  ellipses,  etc.; 

c.  Rotating  a  portion  of  a  graphic  around  a  specific  point; 

d.  Setting  an  object  scale  factor; 

e.  Setting  gray  scale  levels  for  various  portions  of  a  graphic;  and 

f.  Drawing  dashed  lines  of  various  patterns. 

3.  Text  Editor.  A  text  editor  with  the  following  capabilities  is  needed: 

a.  Utility  text  editing; 

b.  Establishing  backward  and  multiple  forward  pointers; 

c.  Establishing  reference  pointers  to  a  graphic  file;  and 

d.  Compressing  and  converting  data  into  a  byte  string. 

4.  Applications  Program.  Applications  software  for  presenting  technical 
data  is  needed  with  the  following  capabilities: 

a.  Checking  whether  a  technician  has  access  rights  to  the  material 
requested  and  denying  access  If  not  authorized. 

b.  Presenting  Information  requested  by  the  technician. 

c.  Collecting  relevant  data  for  performance  analysis  and  maintenance 
data  collection  requirements. 


Discussion  and  Conclusions 


The  feasibility  of  developing  an  automated  technical  data  system  was 
demonstrated.  The  success  of  such  a  system  is  dependent  upon  whether  it  fully 
meets  the  information  needs  of  the  technicians  who  will  use  the  system.  A 
sound  human  factors  impact  analysis  Is  essential  for  building  such  a  system  to 
ensure  its  usability  and  acceptability. 

The  major  needs  of  the  maintenance  technician  must  be  considered  when 
developing  an  automated  technical  data  presentation  system.  If  a  sound  human 


30 


factors  impact  analysis  is  not  done  before  building  such  a  system,  the 
resultant  system  will  have  limited  usability  and  acceptability. 

The  availability  of  suitable  hardware  and  software  is  a  prerequisite  of 
developing  such  a  system.  Knowledge  of  current  hardware  (as  of  1979)  and 
anticipated  developments  in  hardware  indicate  that  the  development  and 
deployment  of  an  automated  job  performance  aiding  system  in  the  near  future  is 
feasible.  In  the  Immediate  future,  portability  will  be  a  problem  due  to  the 
fact  that  solid-state  mass  memory  devices  are  not  available.  Thus,  reliance 
must  be  placed  on  magnetic  disk  random  access  storage.  Similarly,  flat-panel 
technology  has  not  reached  the  point  of  providing  sufficient  graphics 
resolution.  CRTs  appear  to  be  the  best  interim  solution. 

The  study  demonstrated  that  the  software  support  required  for  an  automated 
job  performance  aiding  system  can  be  developed  at  this  time  (1979).  However, 
new  software  development  will  be  required  to  streamline  the  data  preparation 
and  retrieval  processes.  Specific  areas  requiring  software  development 
include  applications  software  for  development  of  graphic  materials  and 
software  to  control  the  retrieval  and  presentation  of  data  to  the  technician. 

Technician-oriented  considerations  are  the  most  Important.  Many  features 
of  the  best  hard-copy  job  performance  aids  are  good  starting  points  for 
developing  principles  for  automated  technical  data  design.  Issues  such  as 
graphics  simplification,  having  all  of  the  Information  to  do  the  job  at  hand, 
and  alternate  levels  of  technical  detail  have  their  basis  In  paper-based  job 
performance  aids  and  are  readily  adaptable  to  automated  aids. 

All  of  the  automated  technical  data  presentation  concepts  developed  and 
tested  in  these  studies  seem  worthy  of  further  Investigation.  Specifically, 
multiple  levels  of  detail  (tracks),  the  function  keyboard,  graphics 
simplification,  and  pools  of  auxiliary  Information  seem  to  have  the  most 
promise  as  design  concepts. 

Some  Issues  not  Investigated  were  considered  Important  to  the  success  of 
an  automated  system.  The  amount  and  nature  of  the  Interchange  between  the 
system  operator  and  the  system  required  additional  study  (and  still  do). 
Several  design  Issues  in  this  area  required  study.  Including  the  desirability 
of  having  the  computer  acknowledge  each  request  and  the  desirability  of 
requiring  the  technician  to  indicate  that  he  understands  the  Instruction. 
Similarly,  study  was  (and  is)  needed  to  determine  the  Impact  of  format 
considerations  and  to  determine  which  formats  and  cuing  techniques  were  the 
most  effective. 


III.  INITIAL  PROTOTYPE  DEVELOPMENT  EFFORT 

In  September  1978,  a  contract  was  awarded  to  Unified  Industries,  Inc. 
( U 1 1 )  for  the  development  of  a  full-scale  prototype  CMAS  for  intermediate 
level  (shop)  maintenance.  Contractual  arrangements  Included  a  subcontract  to 
BioTechnology,  Inc.  for  development  of  the  MMJ,  formats,  and  the  human  factors 
aspects  of  the  system  design.  This  work  was  based  on  the  feasibility  studies, 
and  extended  and  refined  the  basic  concepts  developed  in  those  studies. 


Work  continued  on  the  development  of  the  prototype  until  March  1981,  when 
It  was  terminated  at  the  convenience  of  the  Government.  Contract  termination 
became  necessary  when  a  change  In  requirements  for  the  system  was  received 
from  the  project  sponsor,  the  Air  Force  Logistics  Command.  The  change  added 
the  requirement  that  the  system  must  be  suitable  for  deployment  In  support  of 
worldwide  wartime  operations.  The  system  under  development  was  designed  for 
use  at  fixed  site  locations  and  was  not  suitable  for  deployment.  It  was 
determined  that  It  would  not  be  cost  effective  to  modify  the  system  to  make  It 
deployable.  Thus,  the  contract  was  terminated. 

During  the  period  of  the  contract,  work  was  accomplished  on  several  tasks 
In  preparation  for  the  design  and  development  of  the  system.  The  work 
accomplished  during  this  period  provided  an  essential  background  for  follow-on 
efforts  to  build  a  prototype  CMAS.  The  basic  hardware  (computer  system  and 
display)  for  the  prototype  system  was  procured.  However,  no  software  was 
developed.  The  major  areas  of  work  accomplished  Included: 

1.  Identification  of  technicians4  Information  requirements; 

2.  Development  of  the  MMI; 

3.  Development  of  data  presentation  formats; 

4.  Development  of  task  analysis  procedures  appropriate  for  the  develop¬ 
ment  of  technical  data  for  automated  presentation; 

5.  Identification  of  software  requirements;  and 

6.  Identification  of  hardware  requirements. 


Information  Requirements  Analysis 

A  detailed  analysis  was  made  to  Identify  the  types  of  Information  that 
must  be  provided  by  an  automated  technical  data  system.  This  analysis  built 
upon  the  user  analysis  developed  by  Frazier  et  al.  (1979),  upon  the  earlier 
job  performance  aids  research,  and  upon  the  Information  requirements 
established  by  existing  technical  data  specifications.  The  latter  source  was 
examined  to  ensure  that  the  CMAS  is  capable  of  providing  all  types  of  data 
required  to  maintain  a  system.  In  addition,  an  analysis  was  made  of  existing 
techniques  for  presenting  technical  Information  via  automated  systems  and  for 
presenting  Improved  technical  data  on  paper.  The  latter  emphasized  the  work 
of  AFHRL  and  other  research  agencies  to  develop  Improved  Job  performance 
aids.  Work  that  addressed  ways  of  matching  technical  data  to  the  skills  and 
needs  of  the  user  was  examined  also. 


Man/Machine  Interface 

The  majority  of  the  work  accomplished  was  In  the  areas  of  the  MMI  and  the 
development  of  technical  data  presentation  formats.  Since  the  contract  was 
terminated  before  the  prototype  could  be  built,  only  the  MMI  and  technical 
data  format  designs  were  completed.  The  major  characteristics  and  features  of 


the  MM I  design  and  presentation  format  designs  are  presented  In  the  following 
paragraphs.  The  Materials  presented  are  extracted  fro*  Hatterlck  (1985)  and 
Thomas  (1982). 

The  primary  objectives  of  the  MMI  design  were  to  provide  a  system  that: 

1.  Is  easy  to  use  and  requires  no  special  skills  to  operate; 

2.  Provides  rapid  access  to  the  desired  data; 

3.  Provides  for  easy  movement  within  the  data  base; 

4.  Enhances  the  performance  of  the  maintenance  technician;  and 

5.  Receives  positive  user  acceptance. 

To  meet  these  object! ves,  the  MMI  provides  for  multiple  levels  of  detail 
(tracks),  pools  of  support  Information,  menu  access  to  data,  direct  access  to 
data,  and  manipulation  of  graphics.  A  special  function  panel  Is  provided  for 
Input  of  requests  to  the  computer. 


Function  Panel 

The  principal  component  of  the  MMI  design  Is  the  special  function  panel/ 

keyboard  which  Is  similar  to  that  proposed  by  Frazier  et  al.  The  panel 

(Figure  12)  Is  designed  to  make  It  easy  for  the  user  to  find  the  Information 
that  he  desires  and  to  move  freely  within  the  data  base.  The  panel  Is 

designed  to  make  It  possible  for  technicians  to  accomplish  the  following 

operations  with  one  or  two  keystrokes: 

1.  Display  the  next  frame  In  sequence  (FORWARD  key); 

2.  Return  to  the  previous  frame  (REVERSE  key); 

3.  Display  a  list  of  available  optional  Information  (LIST  OPTIONS  key); 

4.  Return  (to  a  procedural  frame)  from  a  pool  Item  (RETURN  key); 

5.  Change  scale  to  make  a  graphic  larger  or  smaller  (LARGER  and  SMALLER 

keys).  A  RESTORE  key  returns  the  graphic  to  its  original  scale; 

6.  Scroll  or  pan  a  graphic  to  view  a  different  portion  of  the  graphic  for 
graphics  larger  than  the  display  window  (arrow  keys); 

7.  Store  a  frame  of  data  for  immediate  recall  (HOLD  key  to  store,  SHOW 
key  to  display); 

8.  Select  data  access  mode  (USER  REQUEST  key).  This  function  allows  the 

user  to  change  data  access  modes  from  the  default  mode  (menu  access)  to  the 

user  request  mode  and  vice  versa. 

9.  Enter  YES  or  NO  response  (Y  *  yes,  N  *  no); 
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12.  Function  Panel  Layout 


10.  Enter  alphanumeric  characters  for  direct  access,  entry  of  user 
Identification  number,  etc.; 

11.  Turn  the  system  on  or  off  (START  key  or  STOP  key); 

12.  Jump  forward  to  the  next  major  node  In  the  preestablished  sequence 
(NEXT  SEQUENCE  key);  and 

13.  Jump  backward  to  the  next  major  node  in  the  predetermined  sequence 
(REPEAT  SEQUENCE  key). 

The  Initial  mechanical  design  for  the  panel  provides  for  construction  as  a 
sealed,  liquid  proof  unit.  Membrane  keys  covered  with  plastic  are  used  for 
the  keyboard.  Function  keys  on  the  panel  are  Identified  by  printing  key 
Identifications  on  an  overlay.  This  approach  provides  maximum  flexibility  In 
that  the  function  key  layout  of  the  panel  can  be  changed  by  simply  changing 
the  overlay  and  modifying  the  software.  The  design  provides  for  connecting 
the  panel  to  the  computer  system  by  a  cable  long  enough  to  allow  the  user  to 
move  the  panel  to  any  location  In  his  work  area  within  viewing  distance  of  the 
display.  The  contract  was  terminated  before  the  panel  could  be  constructed. 


Data  Access  Methods 

The  MMI  design  provides  for  four  methods  of  accessing  maintenance  data: 
from  a  standard  mode  (menu-based  access),  by  a  direct  access  mode,  from 
Internal  branching  within  the  technical  data  Itself,  and  from  a  list  of 
options  accessed  from  within  the  technical  data  itself.  Illustrated  parts 
breakdown  (IPB)  Information  presents  some  unique  data  access  problems  since 
the  data  must  be  accessed  using  a  wide  variety  of  locator  Information  (part 
number,  reference  designator,  etc.)  and  must  be  used  for  special  purposes 
(e.g.,  supply).  The  design  provides  for  the  use  of  a  combination  of  the  above 
approaches  to  access  IPB  Information.  The  methods  provided  by  the  MMI  design 
for  accessing  each  type  of  data  are  briefly  described  below. 

Standard  Access.  The  standard  access  mode  provides  access  to  technical 
data  from  a  series  of  menus  or  tables  of  contents  which  progressively  narrow 
the  choices  until  the  desired  Item  Is  located.  The  successive  menus  provide 
for  the  selection  of  the  appropriate  TO  (If  the  data  base  Includes  more  than 
one  TO),  Identification  of  series  (A-Model,  B-Model,  etc.)  of  the  equipment  to 
be  worked  on,  selection  of  the  system,  selection  of  the  subsystem,  selection 
of  the  component,  and  selection  of  the  specific  procedure  or  other  Information 
to  be  retrieved.  Selections  from  a  menu  are  made  by  entering  the  Identifying 
number  of  the  item  and  pressing  the  ENTER  key.  The  number  of  menus  required 
to  identify  a  specific  item  of  information  primarily  depends  -upon  the  size  and 
complexity  of  the  TO  and  the  level  within  the  system  of  the  Item  of  interest 
(e.g.,  fewer  menus  are  required  to  locate  a  system  checkout  procedure  than  to 
locate  the  procedure  to  remove  a  lower-level  component  of  the  system). 

User  Request  Method.  The  user  request  method  provides  the  technician  with 
a  means  of  going  directly  (direct  access)  to  a  specific  procedure  or  data 
element  without  going  through  the  cumbersome  menu  selection  process.  After 
system  sign-on  or  at  any  point  In  any  procedure,  the  technician  can  activate 


the  direct  access  Mode  by  pressing  the  USER  REQUEST  function  key  on  the 
panel.  The  systea  then  proapts  the  user  to  enter  the  specific  request.  Using 
the  alphanuaerlc  keys,  the  technician  can  retrieve  a  specific  fraae  or  Itea  of 
Inforaatlon  provided  that  he  knows  one  of  the  following  pieces  of  Inforaation: 

Fraae  Nuaber 
Task  Nuaber 
Fault  Code 

Maintenance  Inforaatlon  Oata  Access  Systea  (MIDAS)  Code 
Part  Nuaber 
Reference  Designator 

Options  Menu  Access.  At  any  point  In  a  procedure,  the  user  can  press  the 
LIST  OPTIONS  function  key.  The  systea  responds  with  a  list  of  options 
available  at  that  point.  The  options  list  aay  include  one  or  more  of  the 
following  types  of  options: 

Change  Tracks  (aore/less  detail)  ' 

Pool  Inforaatlon 

How  to  use  required  test  equlpaent 

Functional  diagrams 

Schematics 

Theory  of  operation 

System  descriptions 

Record  of  applicable  Time  Compliance  Technical  Orders  (TCTOs) 

Parts  Information 

User  record  (sequence  of  steps  followed  by  user) 

Lists  of  tolerances  and  test  specifications 
Glossaries 

Test  equlpaent  and  tool  use  descrlptlons/lnforaatlon 
Return  to  Table  of  Contents 

List  of  effective  frames  (equivalent  to  list  of  effective  pages  In  TO) 

Help  (how  to  use  the  systea) 

Quit  procedure 

Internal  Branching.  The  system  design  provides  the  capability  to  go 
directly  from  one  procedure  to  another  In  certain  cases.  This  capability  Is 
provided  In  the  following  cases: 

1.  Another  task  must  be  performed  prior  to  the  current  task  (e.g., 
aircraft  must  be  made  safe  for  maintenance). 

2.  A  follow-on  task  must  be  completed  to  return  the  system  to  operational 
condition. 

3.  A  fault  Identified  In  troubleshooting  must  be  remedied  before  the 
system  can  become  operational. 

When  the  above  situations  occur,  the  systea  displays  a  prompt  asking  the 
technician  whether  he  wants  the  procedure  required  for  the  new  task. 

IPS  Information  Access.  The  design  provides  both  standard  and  direct 
access  methods  7or  accessing  IPS  Information.  The  strategies  for  locating  IPS 
information  are  presented  diagrammatical ly  In  Figure  13. 


In  using  the  standard  access  method,  the  user  Is  led  through  a  series  of 
menus  which  help  him  take  the  information  that  he  has  about  the  part  of 
Interest  to  narrow  the  choices  until  he  locates  the  desired  information.  The 
primary  menus  used  are: 

Part  Number  Index 

Major  Assembly  List 

Reference  Designator  Index 

One  or  more  submenus  may  used  to  further  narrow  the  choices  until  a  frame  with 
the  desired  information  Is  located. 

In  the  user  request  mode,  the  user  may  go  directly  to  a  composite  parts 
breakdown  frame  which  provides  specific  Information  on  the  part  or  subassembly 
of  Interest.  This  design  feature  makes  It  possible  for  the  knowledgeable  user 
to  eliminate  many  Intermediate  steps. 

In  addition  to  the  standard  access  and  the  user  request  modes,  parts 
Information  often  can  be  obtained  directly  from  the  options  list  (provided  It 
Is  a  listed  option  at  that  specific  point  In  the  procedure). 


Multiple  Track  Definitions 

The  multiple  levels  of  detail  (multiple  track)  concept  proposed  by  Frazier 
et  al.  (1979)  was  adopted  for  the  prototype  system.  Technical  data  were  to  be 
provided  In  three  levels  of  detail  or  tracks.  The  three  tracks  and  the 
Intended  users  are  described  In  the  following  paragraphs; 

Track  3.  This  track  Is  Intended  for  the  novice  technician.  The  novice  Is 
described  as  having  a  general  understanding  of  the  system  or  similar  systems, 
but  having  only  limited  specific  knowledge  of  the  system.  It  Is  assumed  that 
he  Is  not  familiar  with  specific  system  components  or  their  location  and, 
therefore,  requires  assistance  In  locating  them.  Also,  he  Is  unfamiliar  with 
the  procedures  required  to  perform  specific  tasks.  Thus,  specific  step-by- 
step  Instructions  are  required  for  him  to  perform  them.  It  Is  assumed  that 
the  novice  has  a  basic  understanding  of  the  use  of  any  required  special  tools 
and  test  equipment,  but  Is  not  fully  proficient  In  using  them.  It  Is 
anticipated  that  this  track  will  be  used  primarily  by  3-skl  11-level 
technicians.  However,  It  may  be  appropriate  for  some  5-skl  11 -level  and 
7-sk111-1evel  technicians  who  are  completely  unfamiliar  with  the  specific 
system  or  the  assigned  task. 

Track  2.  This  track  Is  designed  for  the  journeyman  technician.  The 
journeyman  is  described  as  a  fully  qualified  5-ski11-1evel  technician  with  at 
least  6  months  of  experience  on  the  system.  The  journeyman  Is  thoroughly 
familiar  with  the  system  and  has  accomplished  most  commonly  performed  tasks  on 
the  system  at  least  a  few  times.  He  knows  the  layout  of  the  equipment,  can 
identify  Its  components  without  aid,  and  Is  able  to  perform  routine  trouble¬ 
shooting  tasks  on  the  system  with  the  aid  of  predeveloped  troubleshooting 
strategies.  It  Is  assumed  that  the  technician  is  proficient  In  the  use  of  any 
special  tools  or  test  equipment.  It  Is  anticipated  that  this  track  will  be 
used  primarily  by  fully  qualified  5-sklll-level  technicians.  However,  It  may 


be  appropriate  for  7-ski  11-level  technicians  who  have  not  worked  on  the  system 
for  some  time,  who  are  transferring  from  another  weapon  system,  or  who  are 
working  on  a  specific  task  that  they  have  not  performed  for  some  time.  Also, 
Track  2  data  may  be  appropriate  for  some  3-skill-level  technicians  who  have 
performed  the  specific  task  many  times. 

Track  1.  This  track  is  designed  for  use  by  the  "expert."  The  expert  is 
described  as  a  technician  with  extensive  experience  on  the  system  being 
maintained,  and  extensive  knowledge  of  the  system  and  how  it  operates.  He  is 
able  to  perform  most  tasks  with  only  limited  technical  data  to  remind  him  of 
critical  actions  or  needs  only  specific  information  such  as  tolerances.  This 
individual  is  capable  of  developing  his  own  strategies  for  troubleshooting  the 
system  using  aids  such  as  schematics.  It  is  assumed  that  the  individual  Is 
proficient  in  the  use  of  any  required  tools  or  test  equipment.  Normally,  it 
Is  anticipated  that  this  track  will  be  used  by  7-skill-level  and  senior 
5-skl 11-level  technicians.  However,  there  may  be  Instances  in  which  junior 
5-skl 1 1 -level  and  some  3-skl 1 1-level  technicians  may  be  experts  on  one  or  more 
specific  tasks  (which  they  have  performed  many  times).  In  these  instances,  it 
may  be  appropriate  for  them  to  use  Track  1  data  (with  their  supervisor's 
approval ) . 

It  should  be  noted  that  Implicit  In  the  above  definitions  is  the 
philosophy  that  the  track  level  used  should  not  be  based  upon  the  “official" 
skill  level  (3-,  5-,  or  7-skl  1 1  -level )  but  on  the  technician's  knowledge  of 
the  specific  task  to  be  performed.  Thus,  simply  because  a  technician  Is  a 
7-ski  11 -level.  It  does  not  follow  that  he  should  always  use  Track  1  data,  or 
that  a  5-skl 11 -level  technician  should  always  use  Track  2  data.  It  is 
possible  for  a  5-skl  11-level  to  be  an  expert  on  a  given  task.  In  that  case. 
It  would  be  appropriate  for  him  to  use  Track  1  data.  Similarly,  it  would  be 
possible  for  a  7-sklll-level  technician  to  be  completely  unfamiliar  with  a 
given  task.  In  this  case,  the  use  of  Track  2  or  even  Track  3  data  would  be 
appropriate. 


I  Format  Development 

Baseline  requirements  for  the  presentation  of  technical  data  were  based 
upon  materials  from  MIL-M-3880QA,  MIL-M-83495,  Lobel  and  Mulligan  (1980), 
Mulligan  (1980),  and  Mulligan  and  Bird  (1980).  Formats  were  developed  for 
each  specific  type  of  Information  identified  for  inclusion  in  the  CMAS.  The 
formats  can  be  generally  classified  Into  three  categories:  formats  for 
maintenance  procedures,  formats  for  troubleshooting  procedures,  and  formats 
for  support  Information.  Formats  were  developed  for  presenting  maintenance 
procedures  and  troubleshooting  procedures  in  three  tracks  appropriate  to  the 
definitions  specified  above.  Formats  for  nonprocedural  support  materials  are 
provided  in  one  level  of  detail.  The  formats  developed  are  described  briefly 
below.  Detailed  descriptions  of  the  formats  are  presented  In  Hatterick  (1985). 

[  Formats  for  Maintenance  Procedures 

i 

■  Maintenance  procedures  were  defined  to  include  all  procedural  information 

that  does  not  Include  troubleshooting,  checkout,  fault  isolation,  and  test. 

[  Formats  were  developed  which  were  appropriate  for  the  presentation  of  tasks 

for  the  following  activities: 


t 

•V»» 

iYj 


Inspection 

Cleaning 

Disassembly  or  removal 
Assembly  or  installation 
Lubrication 

Alignment,  adjustment,  and  calibration 

Preoperational  check 

Repair 

The  formats  for  each  track  have  the  following  characteristics: 

Track  3.  Track  3  procedures  provide  the  most  detail.  Each  procedure 
contains  the  following  types  of  information: 

1.  Input  Conditions.  The  input  conditions  provide  the  information 
required  to  prepare  to  perform  the  task.  The  following  categories  of 
information  are  provided: 

Applicable  serial  numbers 

Personnel  required  (number  and  specialty) 

Supplies 

Special  tools  and  test  equipment 

Equipment  conditions  (including  instructions  for  establishing  conditions 
and  requirement  to  verify  condition  prior  to  accessing  next  frame) 
Summary  of  procedures 

(See  Figures  14,  15,  and  16.) 


«  TO  1C-141A-2-AA  >mmi  ws  iwmc 

#  C-141A  INSINE  FUEL  SYSTEM  MAINTENANCE 

i  task  4-i:  run  roar  mm  element  mom  i  installation,  i 


INPUT  CONDITIONS: 

•  APPLICABLE  SENIAl  NUNIERS. . 


REMOVE  INSTALL 


«  •  PERSONNEL  REQUIRED .  1 

«  -  SPECIALIST  HILL  IE  REQUIRED  TO  MOTOR 

»  AFFECTED  EN6INE  UPON  REQUEST  DURIN6  FUEL 

#  SYSTEM  ACTIVATION  I  CHECKOUT. 

•  SUPPLIES: 

#  •  ONE  SALLON  PLASTIC  CONTAINER .  1 

#  -  PETROLATUM  IREASE,  VV-P-2J6 . 

#  •  O-RING  PACKING,  MSM21-13S . 

#  -  HIRING  CLOTH  OR  TOWELS  (HIRES) .  V 

•  SPECIAL  TOOLS  AND  TEST  EQUIPMENT: 

#  -  MAINTENANCE  STAND,  TYPE  I-4A  (ALT:I-1A)  1 

t  -  TORQUE  HRENCH,  Slf  lNCH-POUNDS(CALllRATEO)- 


D  IR^UT  CONDITIONS  CONTINUED:  [FORHARD]. 


Figure  14.  Exanple  of  Maintenance  Task  Input 
Conditions  Frame  (All  Tracks). 
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TO  1C-141A-I-AA  7J-1I-M 

4-3:  INPUT  CONOITIONS.  CONTINUED. 

•  EQUIPMENT  CONDITIONS: 


SOS  D3S00D 


*************************  MANN I NO  44444444444444444444444444 
THE  FOLLOWING  EQUIPMENT  CONDITIONS  MUST  DE  MET  DEPONE 
DO I NO  THE  TASK  YOU  HAVE  SELECTED. 

444444444444444444444444444444444444444444444444444444444444 

•  HAS  FUEL  FNOM  TANKS  DEEN  SHUT  OFF  FON  APPLICASLE  END1NE 

BAS  SHOWN  |Y: 

•  FINE  EMENSENCV  HANDLE  *1*  PULLED  AND  TADDEO? 

•  ENDINE  FINE  EXT1NDUISHEN  CINCUIT  DNEAKEN  *2*  OPEN  AND 
f  TADDEO? 


*1*  TYPICAL 


i 


typical 


#  ©  ANE  DOTH  CONDITIONS  YEN1FIED?  INPUT  [YES]  ON  [NO]  [  ); 


Figure  15.  Example  of  Maintenance  Task  Input  Conditions 
Continuation  Frame  (Tracks  2  and  3). 


I  TO  1C-141A-2-AA  73-10-00 

«  4-1:  INPUT  CONDITIONS.  CONTINUED 


SOD  03S0SA 


•  SUNHANV  OF  PNOCEOUNE: 

-  ENSUNE  APPLICABLE  EQUIPMENT  IS  IN  THE  CONNECT  CONDITION 
FON  MAINTENANCE. 

-  NENOVE  PUTIN  ELEMENT  FNOM  THE  FUEL  PUMP. 

-  DISASSEMBLE  FUEL  PUMP  FILTEN  ELEMENT. 

-  INSPECT  AND  CLEAN  FUEL  PUMP  FILTEN  ELEMENT. 

-  ASSEMBLE  FUEL  PUMP  FILTEN  ELEMENT. 

-  INSTALL  FILTEN  ELEMENT  IN  THE  FUEL  PUMP. 

-  ACTIVATE  AND  CHECKOUT  FUEL  SYSTEM. 

-  PENFONH  NEQUINED  FOLLOW-ON  MAINTENANCE. 

.•  THIS  PNOCEOUNE  SHOULD  BE  PENFONHED  USIND  THE  SPECIFIC 
PNOCEOUNES  WHICH  FOLLOW. 


8  FON  NEXT  FNAME :  [FONWANO]. 

FON  INSTALLATION  PNOCEDUNES  ONLY:  [NEXT  SEQUENCE). 


Figure  T6.  Example  of  Maintenance  Task  Input  Conditions 
Continuation  Frame  (Tracks  2  and  3). 
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2.  Warning,  Cautions,  and  Notes.  This  information  is  imbedded  in  the 
procedure  immediately  prior  to  the  affected  steps. 


3.  Step-by-Step  Instructions.  Very  detailed  step-by-step  instructions 
are  provided.  Each  step  is  keyed  to  a  diagram  showing  the  location  of  the 
component  referenced.  Each  frame  presents  instructions  for  accomplishing  one 
subtask.  The  subtask  is  described,  with  step-by-step  instructions  for 
accomplishing  it  indentured  below.  Each  component  or  part  referenced  in  the 
text  is  keyed  to  the  illustration.  An  example  of  a  Track  3  procedural  frame 
is  presented  in  Figure  17. 


i  _ 

TO  1C-H1A-2-AA 
4-3:  REMOVE  FUEL 


J- 


73-U-ll 

rump  filtea  element 


336  I3487F 
A 


I 


A.  REMOVE  FILTEA  ASSEM6LT  FROM 
1.  POSITION  CONTAINER  IEL0M 


FUEL  PUMP: 

FILTER  *1*  TO  CATCH  DRAINAGE. 


NOTE 


NIPE  ANY  FUEL  SPILLAGE  FROM  ENGINE  KITH  CLOTH  OR  TOWELS. 

2  CUT  SAFETY  MIRE  ANO  REMOVE  2  ROLTS  *2*  AND  MASHERS. 

3.  WITHDRAW  FILTER  ASSEMBLY  *1*  FROM  CAVITY  IN  FUEL  PUMP 
•3*. 


O  NEXT  FRAME:  [FORWARD]. 


Figure  17.  Example  of  Maintenance  Task  Procedure 
Frame  (Track  3). 


Track  2.  Track  2  procedures  provide  the  following  information: 

1.  Input  Conditions.  The  input  conditions  frames  for  Track  3  and  Track  2 
are  identical  in  most  cases. 

2.  Warning,  Cautions,  and  Notes.  This  information  is  imbedded  in  the 
procedure  immediately  prior  to  the  affected  steps. 

3.  Step-by-Step  Instructions.  The  procedures  for  Track  2  provide  the 
same  basic  steps  as  track  3,  with  much  of  the  detail  removed.  The  text 
description  is  normally  the  same  as  the  subtask  description  provided  in  Track 
3,  without  the  detailed  instructions  for  completing  the  subtask. 
Illustrations  with  keyed  callouts  may  be  used  to  supplement  the  text  depending 
upon  the  requirements  of  the  task  being  performed.  An  example  Track  2 
procedure  frame  is  presented  in  Figure  18. 
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«  TO  73-IR-B*  U521C 

#  4-3:  REMOVE  FUEL  PUMP  FILTER  ELEMENT  S 

« 

f  A.  REMOVE  FILTER  ASSEMBLY  *1*  FROM  FUEL  PUMP  CAVITY  «2*. 


CATCH  DRAINA6E  t  WIPE  SPILLAGE  FROM  ENGINE 


I  B.  DISASSEMBLE  ELEMENT  *3*  FROM  COVER  *4«. 


.  CAUTION  . 

COVER  FUEt  PUMP  OPENING  *2*  KITH  COVER  •**. 


■,VU 


,\D  TYPICAL 

•1*-^  #  2  PLACES 

\* 

o  REMOVE  AND  INSPECT  CONTINUED:  [FORWARD]. 


Figure  18.  Example  of  Maintenance  Procedure  Frame  (Track  2). 

Track  1.  The  Track  1  procedures  provide  the  following  types  of 
information: 

1*  Input  Conditions.  The  input  conditions  for  Track  1  are  basically  the 
same  as  used  for  Track  2  and  3  with  the  exception  that  instructions  for 
establishing  required  equipment  conditions  may  be  somewhat  less  detailed. 

2.  Warnings.  Notes,  and  Cautions.  This  information  is  presented  In  the 
Input  Conditions  with  the  Summary  of  Procedures. 

3.  Procedural  Information.  Expert  maintenance  technicians  require 
limited  procedural  information.  In  most  cases,  the  Summary  of  Procedures  from 
the  Input  Conditions  is  used  to  present  this  information.  In  some  cases,  such 
as  complex  alignments,  additional  procedural  information  may  be  provided.  If 
the  expert  technician  requires  more  information,  it  may  be  obtained  by 
accessing  the  Track  2  data.  An  example  of  a  Track  1  procedural  frame  is 
included  in  Figure  19. 

Formats  for  Troubleshooting  Procedures 


Troubleshooting  procedures  are  also  presented  in  three  levels  of  detail. 
The  criteria  for  establishing  the  levels  of  detail  are  the  same  as  for 
maintenance  procedures.  Troubleshooting  procedures  are  provided  for  the  test 
and  checkout  function  and  the  fault  isolation  function. 
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#  TO  1C-141A-2 -A*  73-1B-BB 

#  4-3:  INFUT  CONDITIONS.  CONTINUED 

I 


US  135**4 


SUMMARY  OF  PROCEDURE: 

•  ENSURE  APPLICABLE  EQUIPMENT  IS  IN  THE  CORRECT  CONDITION 
FOR  MAINTENANCE. 

•  REMOVE  FILTER  ELEMENT  FROM  THE  FUEL  PUMP. 

-  DISASSEMBLE  FUEL  PUMP  FILTER  ELEMENT. 

-  INSPECT  AND  CLEAN  FUEL  PUMP  FILTER  ELEMENT. 

-  ASSEMBLE  FUEL  PUMP  FILTER  ELEMENT. 

-  INSTALL  FILTER  ELEMENT  IN  THE  FUEL  PUMP. 

-  ACTIVATE  AND  CHECKOUT  FUEL  SYSTEM. 

•  PERFORM  REQUIRED  FOLLOH-ON  MAINTENANCE. 


O  IF  YOU  NEED  MORE  INFORMATION:  [LIST  OPTIONS]. 


Figure  19.  Example  of  Maintenance  Task  Input 
Conditions  Summary  Frame  (Track  1). 

The  troubleshooting  formats  used  for  the  prototype  system  are  based  upon 
the  logic  tree  troubleshooting  aid  (LTTA)  as  defined  by  Mulligan  (1980)  and 
MIL-M-38800A.  The  LTTA  Is  composed  of  two  basic  elements:  the  checkout 
procedure  and  the  fault  isolation  procedure.  The  fault  isolation  procedure 
immediately  follows  the  checkout  procedure.  The  user  Is  "branched"  to  the 
appropriate  section  of  the  fault  Isolation  procedure  when  an  out-of -tolerance 
condition  is  Identified  during  the  checkout  of  the  system.  The  formats  for 
each  track  have  the  following  characteristics: 

Track  3.  "Enriched"  LTTAs  are  used  to  present  troubleshooting  procedures 
for  the  most  detailed  track.  The  procedures  and  logic  used  for  Tracks  2  and  3 
are  the  same.  Track  3  provides  more  detail  on  how  to  perform  the  specified 
tests  and  how  to  set  up  and  use  the  specified  checks  and  tests.  The  Track  3 
procedures  include  the  following: 

1*  Input  Conditions.  Separate  Input  conditions  frames  are  provided  for 
the  checkout  and  fault  isolation  sections  of  the  LTTA.  The  Input  conditions 
Information  is  similar  to  that  provided  for  maintenance  procedures.  However, 
there  are  some  exceptions.  The  checkout  Input  conditions  Include  a  summary  of 
the  procedure.  Fault  isolation  Input  conditions  do  not  contain  this 
Information  since  the  sequence  of  steps  is  unpredictable  and  varies  due  to  the 
conditional  branching  applied. 
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2.  Warnings,  Cautions,  and  Notes.  The  rules  for  providing  warnings, 
cautions  and  notes  lor  maintenance  procedures  apply. 

3.  Checkout  Procedures.  The  format  for  presenting  checkout  procedures  is 
the  same  as  for  maintenance  procedures  (see  Figure  20(a)).  Dual -level  subtask 
elements  are  followed  by  detailed  instructions  on  how  to  accomplish  each 
subtask.  The  procedures  are  completely  integrated  with  illustrations  showing 
the  location  of  referenced  components.  Each  subtask  starts  on  a  new  frame. 
Several  frames  may  be  required  to  present  the  complete  subtask.  When 
follow-on  frames  are  used,  the  subtask  is  identified  on  each  frame.  Questions 
are  phrased  so  that  a  YES  response  indicates  a  normal  condition  and  a  NO 
response  indicates  an  abnormal  condition.  An  arrow  placed  at  the  side  of  the 
frame  is  used  as  a  cue  to  indicate  that  an  input  from  the  user  is  required. 
When  the  user  responds  to  a  question,  the  system  replies  with  a  feedback 
message  (Figures  20(b)  and  20(c)).  When  an  out-of -tolerance  condition  is 
Identified,  the  feedback  message  provides  the  option  to  go  to  the  appropriate 
section  of  the  fault  Isolation  procedure. 

4.  Fault  Isolation  Procedures.  Fault  isolation  procedures  are  presented 
in  a  dual-level  format  similar  to  that  used  for  checkout  procedures  (see 
Figure  21).  An  exception  Is  that  the  subtasks  are  enclosed  In  "boxes."  The 
boxes  are  used  to  provide  a  visual  distinction  between  checkout  and  fault 
isolation  procedures.  The  boxes  also  Improve  the  compatibility  between  the 
Track  2  and  Track  3  fault  isolation  procedures.  As  In  the  checkout 
procedures,  arrows  are  used  to  indicate  that  a  user  Input  Is  required. 
Questions  are  stated  so  that  a  YES  indicates  an  In-tolerance  condition  and  a 
NO  response  Indicates  an  out-of -tolerance  condition.  The  system  responds  to 
inputs  in  response  to  a  question  with  a  message  stating  the  meaning  or 
implication  of  the  input  test  results. 

Track  2.  Track  2  troubleshooting  procedures  are  presented  at  the  middle 
level  of  detail  as  defined  In  Mulligan  (1980).  The  Track  2  procedures  contain 
the  following: 

1.  Input  Conditions.  The  Input  conditions  frames  are  the  same  as  Track  3. 

2.  Warnings,  Notes,  and  Cautions.  This  information  Is  presented  the  same 
as  Track  3. 

3.  Checkout  Procedures.  The  sequence  of  procedures  (logic)  Is  the  same 
as  used  for  Track  3  (see  Figure  22).  However,  It  Is  assumed  that  the  user  Is 
sufficiently  familiar  with  the  equipment  and  that  he  knows  where  the 
referenced  controls,  indicators,  and  components  are  located.  Thus, 
illustrations  are  not  provided  to  show  the  location  of  these  items.  It  is 
also  assumed  that  the  user  knows  how  to  perform  standard  tests  and  checks. 
Detailed  instructions  are  not  provided  on  performing  these  actions. 

4.  Fault  Isolation  Procedures.  Track  2  fault  isolation  procedures  are 
presented  In  a  format  very  similar  to  that  used  to  present  LTTAs  in  a  paper 
medium  (see  Figure  23).  Directions  for  accomplishing  a  test  are  presented  in 
a  box  with  arrows  leading  to  the  next  test  (box)  for  in-tolerance  conditions 
and  an  arrow  leading  to  a  branching  instruction  for  out-of-tolerance 
conditions.  For  branching  situations,  the  user  is  required  to  input  the 
appropriate  code  (a,  b,  c,  etc.)  to  cause  the  computer  to  retrieve  the  proper 
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#  TO  10P3-ASQ-I9-2-AA  26-22-03  3FI  I72S4C 

I  CHECKOUT  3-1:  0.  RAH UAL  6R0UHD  SPEED  SLEW  CHECK 

#  1.  CHECK  MEMORY  LAMP: 

#  A.  SET  POWER  SWITCH  *1*  TO  SLEW. 

#  J.  SET  TERRAIH  SWITCH  *2*  TO  LARD. 

#  £T>  DOES  HEMORY  LAMP  *3*  LI6HT  AHO  REGAIN  LIT? 


f  ©  INPUT  EITHER  [YES]  OR  [HO]:  [  ];  [EWTER]. 


T 


INPUT  EITHER  [YES]  OR  [HO]:  [YES];[ENTER]. 

Q  MEMORY  LAMP  IS  OKAY.  CHECK  LEFT  SLEW:  [FORWARD]. 


k 


T 


T 


INPUT  EITHER  [YES]  OR  [NO]:  [NO];  [ENTER]. 
©  MEMORY  CIRCUIT  FAULT.  FOR  LT  3-2:  [FORWARD]. 


k 


T 


T 


Figure  20.  Track  3  Troubleshooting  •  Example  of 
Enriched  Checkout  Frame. 
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#  TO  12PS-CH495-2-AA  29-21-13  3F!  91131C 

#  LT  19-1:  HYDRAULIC  BLOWER  DOES  MOT  SPIN  WHEN  TURNED  ON.  4 


.  CHECK  KOTOR  TO  SLOWER  SHAFT: 

A.  PLACE  MECHANIC'S  STETHESCOPE  ON  OUTLET  FORT  *5*  OF 
MOTOR  *2*. 

B.  SET  MOTOR  SWITCH  *1*  TO  ON. 

C.  LISTEN  FOR  A  COMBINATION  RUMBLING  AND  WHINING  NCISE  OF 
TURNING  MOTOR. 

IS  THE  MOTOR  QUIET? 


INPUT  EITHER  [TES]  OR  [NO]:  LTt»ji  iim 
Q  QUIET  MOTOR  MEANS  SPLINE  SHAFT  IS  OKAY:  [FORWARD]. 


^  INPUT  EITHER  [TES]  OR  [NO]:  [NO];  [ENTER]. 

Q  NOISY  MOTOR  MEANS  SPLINE  SHAFT  1$  RAO.  REPLACE  SPLINE 
SHAFT  WITH  JG  7-5.  FOR  PROCEDURE:  [FORWARD]. 


figure  21.  Track  3  Troubleshooting  -  Example  of 
Enriched  Logic  Tree  frane. 
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I  TO  IHMJ  __  wi  I330M 

*  CMCI8VT  3-1  •.  ».  hunhi  inm  mu  Ken  CHICK.  t 

5  1.  CHICK  WMf  LOHPl  If  HO 

*  «  SIT  IMU  SHITCH  TO  SitH  MO  TltUIK  SNITCH  (  J 
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Figure  22.  Track  2  Troubleshooting  -  Ex«p1e 
of  Checkout  Fraae. 
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Figure  23.  Track  2  Troubleshooting  -  Exwple 
of  Logic  Tree  Fram. 
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troubleshooting  tree.  Instructions  are  presented  with  minimum  detail  based 
upon  the  assumption  that  the  technician  knows  how  to  perform  the  basic  tests. 
Additional  detail  may  be  provided  for  unusual  or  especially  complex  tests. 
Illustrations  showing  referenced  components  are  not  normally  provided. 

Track  1.  The  Track  1  troubleshooting  procedures  provide  minimum  detail. 
The  level of  detail  provided  is  similar  to  that  provided  by  a  standard 
checklist.  The  Track  1  procedures  Include  the  following: 

1.  Input  Conditions.  The  Input  conditions  for  Track  1  are  essentially 
the  same  as  for  Tracks  2  and  3. 


2.  Hamit 
as  for  Tracks 


is.  Notes,  and  Cautions.  This  Information  Is  presented  the  same 
\  and  3. 


3.  Checkout  Procedures.  The  checks  to  be  accomplished  are  listed  In  the 
order  In  which  they  are  to  be  performed  (see  Figure  24).  No  guidance  Is 
provided  on  how  to  accomplish  the  checks  or  where  the  referenced  components 
are  located.  The  only  assistance  provided  Is  an  Indication  of  what  the 
expected  status  or  Indication  should  be.  If  the  normal  status  Is  not  found  or 
an  out-of -tolerance  Indication  Is  found,  branching  to  the  proper  fault 
Isolation  sequence  can  be  obtained  by  entering  the  number  of  the  failed  check. 
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4.  Fault  Isolation  Procedures.  Track  1  fault  Isolation  procedures  are 
presented  in  a  Modified  checkout  format  (see  Figure  25),  This  procedure  Is 
based  upon  the  same  sequence  as  the  logic  tree.  However,  only  the  bare 
essentials  of  symptom  Identification  and  status  Information  are  provided.  The 
possible  faults  are  shown  with  each  symptom.  This  format  is  similar  to 
typical  symptom/cause  charts  found  In  conventional  TOs  and  specified  by 
M2L-M-38800A.  The  difference  Is  that  they  are  based  upon  the  logic  of  the 
LTTA. 
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Figure  25.  Track  1  Troubleshooting  -  Example  of 
Troubleshooting  Oata  Chart  Frame, 


Formats  for  Pool  Data 


Formats  are  required  to  present  a  number  of  different  types  of  pool 
Information  for  presentation  on  the  automated  technical  data  system.  Most  of 
the  data  can  be  accommodated  In  basic  formats  without  special  features.  For 
example.  Theory  of  Operation  Information  Is  primarily  textual  Information  and 
Is  presented  In  a  basic  text  format.  Other  types  of  data,  however,  require 
special  formats.  Primary  examples  are  IPB  Information  and  functional  diagrams 
(e.g.,  schematics  and  block  diagrams).  Morfc  was  accoiqillshed  on  formats  for 
presenting  all  types  of  pool  data.  However,  only  the  formats  for  IPB  and 
block  diagram  formats  are  described  In  this  report.  The  remaining  formats  are 
described  In  detail  In  Hatterlck  (1985). 

IPB  Information.  Formats  for  IPB  Information  are  based  upon  the 
requirements  of  Mll-M- 38807  (USAF)  and  are  designed  to  permit  access  from  the 
same  starting  points  (e.g.,  knowledge  of  assembly  or  subassembly,  part  number, 
reference  designator,  etc.).  Formats  were  developed  for  frames  to  present 
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listings  of  each  type  of  Information  and  for  composite  parts  breakdown  (CPB) 
frames.  The  listing  frames  serve  a  dual  purpose.  They  are  used  to  provide 
specific  Information  (such  as  the  reference  designator  associated  with  a  given 
part)  and  to  present  menus  which  lead  to  the  next  level  of  listing  or  to  the 
CPB  for  an  assembly,  subassembly,  or  part.  The  CPB  frame  provides  detailed 
Information  on  the  subject  part.  The  Information  Includes  the  part  number; 
reference  designator;  Source,  Maintenance  and  Recoverability  (SNR)  code; 
Federal  Supply  Code  for  Manufacturers  (FSCM);  quantity  per  assembly;  Index 
number;  use  on  code;  and  Information  on  the  relation  of  the  part  to  the  next 
higher  and  next  lower  assemblies.  An  illustration  of  the  part  is  also 
provided.  Examples  of  a  CPB  frame  and  a  representative  listing  frame  (part 
number  Index)  are  presented  In  Figures  26  and  27. 


Figure  26.  Example  of  IPS  Part  Numbers  Index  Fr 


x  Formats  for  Functional  Diagrams.  Functional  diagrams  provide  a  problem 
for  presentation  on  a  computer  display  since  they  are  frequently  too  large  to 
present  on  the  screen  In  a  legible  manner.  The  simplest  solution  to  this 
problem  Is  to  present  only  a  portion  of  the  diagram  at  one  time.  However, 
this  presents  a  problem  since  the  user  must  be  able  to  comprehend  the 
relationships  between  the  portion  that  Is  seen  and  the  portions  that  are  not 


seen.  Even  If  the  entire  diagram  Is  provided  and  the  user  Is  able  to  "move" 
the  diagram  to  view  different  portions,  he  Is  likely  to  lose  his  orientation 
and  become  confused. 


Figure  27.  Example  of  IPB  Opposite  Parts  Breakdown  (CPB) 

Frme  (Exploded  View  of  Simple  Mechanical  Assembly). 


The  approach  taken  for  the  prototype  system  to  crercome  the  above  problem 
was  to  use  an  orientation  diagram  to  present  the  overall  diagram  with  the 
basic  functions  and  their  relationships  depicted  (see  Figure  28).  The  diagram 
does  not  provide  detailed  Information  on  any  of  the  functions  shown.  If  the 
user  desires  detailed  Information  on  any  of  the  functions,  he  Inputs  a  code 
from  the  orientation  diagram.  A  detailed  diagram  of  the  function  Is  then 
displayed.  Figure  29  shows  the  diagram  that  would  be  displayed  If  the  user 
entered  the  code  (4)  from  the  diagram  In  Figure  28.  If  the  user  Is  Interested 
In  the  Interface  between  two  functions,  a  different  code  will  call  up  a 
diagram  which  provides  details  on  the  Interface  between  the  functions.  The 
user  Is  able  to  use  the  HOLD/SHOW  functions  to  store  and  quickly  recall  each 
diagram  for  quick  reference. 


% 

J  ■ 
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An  alternative  approach  available  to  the  user  is  to  enter  a  code  from  the 
orientation  diagram  to  obtain  the  complete  basic  diagram.  The  dlagrmn  Is  then 
displayed  in  readable  detail,  and  Is  centered  on  the  function  selected.  The 
user  Is  then  able  to  pan  and  200m  the  display  as  desired. 


Figure  28.  Example  of  Functional  Diagram,  Type  3 

(Orientation  Diagram  of  Complex  Schematics). 


Development  of  Maintenance  Task  Analysis  Procedures 

The  Laboratory's  earlier  work  In  job  performance  aids  research  had 
recognized  the  need  for  a  very  high  degree  of  accuracy  In  procedural  1  zed 
technical  data.  This  research  led  to  the  development  of  Improved  maintenance 
task  analysis  techniques  for  the  development  of  technical  data  to  ensure  that 
the  data  are  accurate.  In  developing  requirements  for  the  prototype  automated 
technical  data  system,  it  was  recognized  that  the  capability  to  develop 
technical  data  with  an  extremely  high  degree  of  accuracy  Is  essential.  It  was 
also  recognized  that  development  of  automated  technical  data  would  provide 
some  unique  problems  since  new  types  of  data  and  data  Interrelationships  would 
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have  to  be  considered.  For  these  reasons,  a  requirement  was  Included  in  the 
contract  to  revise  and  expand  the  task  analysis  procedures  to  support  the 
development  of  accurate  technical  data  for  presentation  on  an  automated  system. 


I  Figure  29. 

i 
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Example  of  Functional  Diagram,  Type  3  (Subfunction 
Diagram  for  High  Complexity  Schematics). 


In  preparation  for  revision  of  the  task  analysis  procedures,  an  analysis 
was  made  of  the  unique  requirements  for  the  development  of  technical  data.  In 
addition,  a  review  was  made  of  existing  task  analysis  procedures  to  determine 
what  changes  were  necessary  to  accommodate  the  new  requirements.  A  draft 
report  describing  the  revised  task  analysis  procedures  was  developed  (Smith  ft 
Hatterick,  1979).  The  contract  was  terminated  before  the  task  analysis 
procedures  could  be  completed  or  tested. 


i 

i 


! 
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Identification  of  Software  Requirements 

Preliminary  work  on  software  requirements  for  the  prototype  system  was 
initiated.  Basic  data  base  management  software  requirements  and  data  access 


strategies  were  studied.  However,  the  contract  was  terminated  before  the 
requirements  analysis  and  design  could  be  completed.  The  work  accomplished 
was  not  formally  documented. 


Identification  of  Hardware  Requirements 

Since  this  research  occurred  very  early  in  the  development  of  automated 

technical  data  systems,  very  limited  guidelines  were  available  on  which  to 

base  the  selection  of  hardware  for  the  system.  A  review  of  available 

guidelines  for  the  development  of  interactive  information  presentation  systems 
was  made  to  identify  known  requirements  for  such  systems.  However,  there  were 
several  areas  (e.g.,  display  resolution  for  graphics)  in  which  there  was 
insufficient  information  to  firmly  establish  hardware  requirements.  Since  it 
was  net  possible  at  that  time  to  establish  firm  hardware  requirements,  a 
decision  was  made  to  procure  equipment  with  more  capability  then  was  expected 
to  be  necessary.  This  approach  would  make  it  possible  to  develop  the 
prototype  system  under  the  "best  conditions"  and,  once  requirements  were 

better  understood,  to  determine  the  minimum  hardware  requirements  for  the 
system. 

Based  upon  the  above  analysis,  hardware  was  selected  and  ordered.  The 
primary  equipment  items  ordered  were  a  VAX  11/780  computer  system  and  a 
Megatek  Model  7000  graphics  terminal.  The  VAX  had  the  capability  to  provide 
more  than  enough  computational  power  to  support  the  prototype  requirements. 
The  Megatek  display  is  a  high-resolution  display  (4096  x  4096  pixels  on  a  12  x 
12  inch  display)  capable  of  displaying  any  graphics  required  for  display  on 
the  prototype  system.  The  computer  and  display  were  delivered  to  the 
contractor.  The  contract  was  terminated  before  the  equipment  could  be 
installed.  The  equipment  was  diverted  to  another  Laboratory  project. 


Discussion 


The  information  provided  above  describes  how  the  prototype  system  was 
designed  to  operate.  However,  since  the  system  was  not  developed,  the 
effectiveness  of  the  design  will  never  be  known.  The  work  accomplished  in 
this  effort  provided  much  of  the  groundwork  for  systems  that  were  developed 
later. 


IV.  CMAS  I  DEVELOPMENT  AND  EVALUATION 

Following  the  termination  of  the  Unified  Industries  effort,  requirements 
for  the  CMAS  program  were  reevaluated  and  the  program  was  restructured.  The 
revised  structure  placed  an  emphasis  on  determining  the  requirements  for  a 
CMAS  system,  including  technical  data  presentation  requirements,  deployment 
requirements,  MMI  requirements,  and  requirements  for  interfacing  with  other 
automated  information  systems.  It  provided  for  the  use  of  existing  hardware 
and  software,  with  some  modifications,  to  develop  a  limited  prototype  to 
demonstrate  and  test  the  concepts.  This  approach  eliminated  the  requirement 
to  develop  hardware  and  software  specifically  for  the  program  as  had  been  the 
approach  in  the  Unified  Industries  effort.  In  addition,  the  B-1B  Program 
Manager  was  in  the  process  of  determining  technical  data  requirements  for  the 

i 
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B- 1 B .  The  use  of  an  automated  technical  data  system  was  under  consideration. 
A  decision  was  made  to  orient  the  revised  CMAS  program  toward  the  B-1B  program 
to  ensure  that  the  specifications  developed  in  the  program  would  meet  the 
requirements  of  the  B-1B. 

A  contract  was  awarded  on  1  December  1982  to  Rockwell  International  with  a 
subcontract  to  Hughes  Aircraft  Company.  The  major  portion  of  the  technical 
work  was  accomplished  by  Hughes  Aircraft  Company.  Rockwell  International 
provided  management  and  expertise  on  the  B-1B  requirements.  Additional 
contractual  support  for  this  effort  was  provided  under  subcontract  by 
BioTechnology,  Inc.  BioTechnology  provided  consultation  in  the  area  of 
technical  data  requirements  and  human  factors.  BioTechnology  had  served  in 
the  same  capacity  on  the  Unified  Industries  effort.  The  results  of  this 
effort  are  discussed  in  this  section. 2 

The  contract  provided  for  the  definition  of  requirements  for  a  CMAS  to 
support  intermediate  level  maintenance,  development  of  a  prototype  system,  and 
development  of  specifications.  The  system  was  to  be: 

1.  Suitable  for  deployment  in  support  of  combat  operations; 

2.  Compatible  with  requirements  for  the  B-1B; 

3.  Compatible  with  the  Automated  Technical  Order  System  (ATOS)  under 
development  by  the  Air  Force  Logistics  Command;  and 

4.  Capable  of  presenting  of  all  types  of  technical  data  required  for 
intermediate  level  maintenance. 

The  approach  taken  by  the  contractors  was  to  first  conduct  detailed 
analyses  of  the  requirements  for  the  CMAS.  The  analyses  included: 

1.  B-1B  maintenance  requirements  analysis; 

2.  Deployment  requirements  analysis;  and 

3.  Human  factors  and  MMI  requirements  analysis. 

One  of  the  objectives  of  the  analyses  was  to  identify  design  issues  for 
which  there  was  not  sufficient  information  available  upon  which  to  base  design 
decisions.  The  next  step  in  the  program  was  to  conduct  a  series  of  small 
design  studies  to  specifically  address  these  issues.  Three  design  studies 
were  conducted.  After  the  design  studies  were  completed,  the  design  of  the 
system  was  accomplished,  the  system  was  fabricated,  and  sample  technical  data 
were  developed  and  input  to  the  system.  The  system  was  then  evaluated  in 
field  tests.  Specifications  were  then  developed  based  upon  the  results  of  the 
field  tests  and  the  earlier  analyses. 


Mhe  work  performed  under  the  contract  is  documented  in  two  unpublished 
reports  by  Hughes  Aircraft  Company  (1985a  and  1985b).  The  materials  presented 
on  pages  57-71  and  pages  73-82  of  the  present  report  are  adapted  from  the 
Hughes  reports.  The  remaining  materials  in  this  section  were  developed  by  the 
authors. 


Based  upon  contract  requirements  and  an  analysis  of  system  goals,  the 
contractors  developed  a  set  of  design  goals.  The  primary  goals  identified  are 
discussed  below. 

1.  Cost.  The  cost  of  developing,  procuring,  and  supporting  the  system 
must  be  minimized  through  the  use  of  available  technologies  (off-the-shelf 
equipment,  etc.)  and  the  use  of  software  designed  for  easy  maintainability. 

2.  Performance.  The  system  must  enhance  user  performance  by  providing 
complete  technical  information  in  a  manner  that  is  matched  to  his  knowledge 
and  capabilities. 

3.  Modular  Design.  The  design  must  be  modular  to  permit  easy  update  with 
new  technological  developments  as  they  become  available. 

4.  Generic  Data  Base.  Data  must  be  maintained  in  a  generic  data  base  to 
allow  presentation  using  different  display  devices  and  formats  without 
changing  the  data  base. 

5.  Multimedia  Capability.  The  system  must  provide  the  capability  to 
present  data  via  multimedia,  including  different  types  of  display  devices  and 
paper. 

6.  Data  Interface  Definition.  The  design  must  provide  for  interfacing 
the  system  with  other  information  systems  such  as  ATOS  and  the  Automated 
Maintenance  System  (AMS). 


Design  Issues 

An  extensive  literature  review  was  conducted  to  identify  data  presentation 
and  MMI  techniques  which  have  been  shown  to  be  effective.  The  search  revealed 
that  the  literature  did  not  contain  adequate  guidance  on  several  design  Issues 
critical  to  the  design  of  the  CMAS.  Among  the  Issues  recognized  were: 

1.  Data  Access  Methods.  The  most  frequently  mentioned  data  access 
methods  were  menus,  fill-in-the-form  or  query,  direct  access  by  data 
identifier,  and  flexible  search.  However,  there  were  no  firm  guidelines  for 
selecting  access  modes  for  various  applications.  Research  was  needed  to 
identify  the  most  effective  data  access  methods  for  automated  technical  data 
systems. 

2.  Presentation  of  Graphics.  There  were  a  number  of  Issues  related  to 
the  presentation  of  graphics  for  technical  data. 

a.  Level  of  Detail.  To  minimize  data  storage  requirements,  it  is 
essential  that  the  amount  of  detail  in  graphics  be  limited  to  the  minimum 
necessary  to  support  the  task.  Adequate  information  was  not  available  to 
permit  an  accurate  definition  of  level  of  detail  requirements  or  to  provide  a 
basis  for  developing  guidelines  to  govern  the  development  of  graphics  for  use 
with  automated  technical  data  systems.  Research  was  needed  to  establish 


requirements  for  the  levels  of  detail  for  various  types  of  graphics  to  be 
presented  via  automated  technical  data  systems. 


b.  Presentation  of  Complex/Large  Graphics.  Graphics  that  are  too 
large  to  be  presented  on  the  display  at  one  time  present  a  special  problem. 
Methods  suggested  for  displaying  graphics  that  cannot  be  displayed  at  one  time 
included:  scrolling,  zooming,  segmentation  of  the  graphic  by  windowing,  and 
storing  the  graphic  as  a  set  of  hierarchical  graphics.  Research  was  needed  to 
determine  which  techniques  are  the  most  appropriate  for  the  complex  graphics 
found  in  technical  data. 

c.  Coding  and  Highlighting.  A  number  of  techniques  have  been 
suggested  for  coding  or  highlighting  text  and  graphics.  These  include  the  use 
of  color,  blinking,  increasing  intensity  of  material  to  be  emphasized,  and 
reverse  video.  Research  was  needed  to  determine  which  of  these  is  most 
effective  for  presenting  technical  data  and  under  which  conditions  they  should 
be  appl ied. 


Design  Studies 

After  examining  the  design  issues  discussed  above  and  after  considering 
the  capabilities  of  the  available  hardware  and  software  and  the  costs  required 
to  add  new  capabilities,  three  design  studies  were  agreed  upon.  Design  Study 
1  was  a  basic  study  to  compare  the  performance  of  technicians  on  representa¬ 
tive  maintenance  tasks  using  an  automated  system  with  their  performance  on  the 
same  tasks  using  the  standard  technical  data.  Design  Study  2  examined  tech¬ 
niques  for  presenting  and  highlighting  complex/large  graphics.  Design  Study  3 
examined  techniques  for  the  presentation  of  text  and  graphics  together.  Dif¬ 
ferent  data  access  methods  were  not  studied  because  the  available  software 
could  support  only  one  access  approach  (menu  access).  The  design  studies  are 
described  below. 

Design  Study  1 

Purposes.  The  purposes  of  Study  1  were: 

1.  To  provide  a  general  demonstration  of  the  features  of  the  proposed 
system;  and 

2.  To  compare  the  performance  of  technicians  using  electronic  and 
conventional  paper-based  technical  data  presentation  methods. 

Method.  Six  Air  Force  technicians  (two  highly  experienced,  two  moderately 
experienced,  and  two  inexperienced)  served  as  subjects.  Each  subject 
completed  two  tasks,  one  with  the  electronic  technical  data  and  one  with  the 
paper-based  technical  data.  The  tasks  were  the  removal  of  a  shop  repairable 
unit  (SRU)  from  the  assembly  and  the  installation  of  the  unit  into  the 
assembly.  Tasks  and  type  of  data  used  were  given  in  a  counterbalanced  order. 
The  removal  task,  whether  performed  with  paper  or  electronic  technical  data, 
was  always  done  first  by  each  subject.  The  electronic  technical  data  were 
presented  on  hardware  and  software  that  were  adapted  from  the  Navy  Technical 
Information  Presentation  System  (NTIPS)  developed  by  Hughes  Aircraft  with  the 
sponsorship  of  the  NTIPS  program  office.  A  black-and-white  display  terminal 
with  512  x  512  pixel  resolution  was  used  to  display  the  data.  The 


Instructions  were  presented  in  a  format  similar  to  that  used  for  Job  Guide 
Manuals  with  step-by-step  instructions  supported  by  illustrations  of  the 
referenced  components  (see  Figure  30).  The  data  were  presented  in  three 
levels  of  detail  (tracks).  The  technician  was  permitted  to  switch  between 
tracks  as  desired. 

The  data  collection  sessions  were  videotaped  so  that  the  technicians' 
performance  and  Interaction  with  the  system  could  be  analyzed.  The  times 
required  to  perform  each  task  and  the  errors  made  were  recorded  during  the 
analysis  of  the  videotapes.  Pre-  and  post-test  questionnaires  were 
administered  to  evaluate  the  technicians'  attitudes  toward  the  paper  and 
automated  technical  data  systems. 

Results.  The  task  selected  for  the  study  was  judged  by  AFHRL  observers  to 
be  too  simple  to  provide  an  adequate  evaluation  of  the  system.  Due  to  this 
problem  and  the  small  number  of  observations,  a  formal  analysis  was  not  made 
of  the  time  and  error  data.  Technical  errors  were  very  few  and  were  not 
analyzed.  However,  it  was  observed  that  the  time  to  perform  each  task  using 
the  electronic  system  was  less  than  the  time  using  the  paper-based  technical 
data.  Analysis  of  the  pre-  and  post-test  questionnaire  data  indicated  that, 
after  the  study,  the  technicians  responded  with  more  positive  responses 
(statistically  significant  at  the  £  <  .05  level)  to  three  of  the  items.  They 
were:  "A  computer-aided  system  would  help  me  do  my  job  faster."  "A  small 
compact  computer  would  be  useful  In  my  work."  and  "A  rugged  computer  package 
would  be  useful  In  my  work."  There  were  more  negative  than  positive  responses 
for  some  questions.  However,  none  of  the  differences  was  statistically 
significant. 

Recommendations.  The  following  recommendations  were  made,  based  upon  the 
results  of  the  study  (Hughes  Aircraft  Company,  1985b,  p.  3-11): 

1.  Pursue  the  development  of  the  CMAS  prototype,  provide  a  greater  range 
of  display  capabilities,  and  test  the  system  using  more  complex  experimental 
tasks. 

2.  Retain  the  three-track  Job  Guide  format  with  Interactive  branchinc 
capability. 

3.  Retain  the  basic  means  of  Interacting  with  the  system  (i.e.,  NEXT  and 
BACK  keys,  menu-driven  selection  of  data,  text  and  graphic  windows  for  data 
presentation,  an  electronic  terminal  and  keyboard  for  data  entry). 


Design  Study  2 

Purpose.  The  purpose  of  Design  Study  2  was  to  identify  optimal  methods 
for  presenting  large,  complex  graphics  via  the  electronic  medium. 

Variables.  Four  variables  were  examined  in  the  study: 

1.  Color-Coding.  Individual  functional  segments  (i.e.,  groups  of  lines 
or  symbols  representing  an  electronic  function)  were  Identified.  Each 
functional  segment  could  then  be  displayed  In  color  or  in  black  and  white  with 
highlighting.  The  color  condition  consisted  of  displaying  the  functional 
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segments  In  different  colors  (up  to  16  distinguishable  colors).  In  the 
black-and-white  condition,  the  diagram  was  Initially  shown  using  solid  lines 
(black  on  white).  By  activating  the  pick-and-highllght  function,  the  user  was 
able  to  select  a  line  or  component.  All  of  the  lines  and  sjm&ols  associated 
with  that  element  were  then  displayed  with  a  dotted  line. 

2.  Resolution.  The  resolution  of  the  display  Is  described  In  terms  of 
the  number  of  pixels  which  can  be  addressed  across  the  display.  Two  levels  of 
resolution  were  evaluated:  high  resolution  (1024  x  1024  pixels)  and  low 
resolution  (512  x  512  pixels).  The  same  display  was  used  for  both  conditions. 
Software  was  used  to  control  the  resolution  level.  The  display  size  was  11.6  x 
15.1  Inches.  The  low-resolution  level  was  typical  for  displays  In  use  at  that 
time. 

3.  Segmentation  Method.  Segmentation  method  refers  to  the  method  of 
breaking  up  large  graphics  Into  smaller  elements  which  can  be  displayed  at  one 
time.  Two  methods  were  evaluated: 

a.  Spatial  Segmentation.  With  spatial  segmentation,  the  graphic  Is 
represented  as  a  two-dimensional  surface.  The  user  can  move  a  "window”  (the 
display  or  portion  of  the  display)  over  the  surface  allowing  him  to  view 
portions  of  the  graphic.  The  effect  Is  similar  to  that  of  using  a  paper 
foldout  where  each  unfolding  reveals  a  portion  of  the  drawing  not  previously 
seen.  See  Figure  31  for  a  representation  of  the  spatial  segmentation  approach. 

b.  Functional  Segmentation.  In  this  method,  the  system  Is 
represented  by  a  series  of  hierarchical  diagrams  based  upon  the  functions  of 
the  components.  The  highest  level  diagram  depicts  the  basic  functions 
Involved  In  the  system.  Each  successive  diagram  provides  a  more  detailed 
breakdown  of  one  or  more  of  the  functions  shown  on  the  higher  level  diagram. 
As  many  layers  as  necessary  may  be  used  to  break  the  functions  down  to  their 
most  basic  structures.  The  diagrams  are  formatted  to  be  viewable  as  a  whole 
on  the  display.  The  user  must  view  several  screens  to  see  the  complete 
Information  at  the  detailed  level.  This  approach  Is  known  as  "pyramiding"  and 
Is  similar  to  that  used  for  paper  technical  manuals  where  foldouts  are  not 
desired.  The  pyramiding  approach  Is  depicted  In  Figure  32. 

4.  Graphic  Interaction  Hode.  The  graphic  Interaction  mode  refers  to  the 
type  of  commands  fKe  user  can  use  to  manipulate  the  display.  Three 
interaction  modes  were  provided: 

a.  Baseline  mode.  In  this  mode,  the  user  was  allowed  to  scroll  and 
zoom  selected  portions  of  the  display.  The  user  was  able  to  select  an  area  of 
interest  (e.g.,  a  rectangle  defined  by  diagonal  corners  Indicated  by  the 
cursor)  on  the  display.  The  system  would  then  expand  the  diagram  to  fill  the 
screen. 

b.  P1ck-and-H1gh11ght  Mode.  In  this  mode,  the  user  was  able  to  move 
the  cursor  to  a  selected  fine  or  symbol  on  the  diagram  and  highlight  the 
associated  functional  segment  by  pressing  a  key  on  the  puck  (or  mouse).  If 
the  color  condition  was  in  effect,  the  selected  segment  would  be  displayed  In 
red.  If  the  black-and-white  condition  was  In  effect,  the  segment  was  shown  In 
dotted  lines.  The  scroll  and  zoom  functions  were  also  available  under  this 
condition. 


Functional  Segmentation  (Pyramid)  of  Schematic  Information 
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c.  Schema  Mode.  In  this  mode,  the  display  Mas  divided  into  two 
windows  (upper  half  and  lower  half).  A  functional  block  diagram  of  the  system 
represented  by  the  schematic  was  presented  In  the  upper  window,  and  the 
schematic  was  presented  In  the  lower  window.  The  zoom,  scroll,  and  plck-and- 
hlghlight  functions  were  available  for  the  schematic  (lower  window)  but  not 
for  the  functional  block  diagram  (top  window)  due  to  limitations  In  the  test 
system  software. 

Experimental  Design.  A  split-plot  factorial  design  was  used.  The  color 
variable  was  the  between-subjects  variable.  The  resolution,  segmentation,  and 
graphic  Interaction  conditions  were  fully  crossed  and  were  presented  as 
repeated  measures  to  all  subjects.  The  order  of  the  experimental  conditions 
was  randomized  for  the  graphics  Interaction  conditions.  The  resolution 
variable  was  presented  In  counterbalanced  blocks.  Six  experienced  Air  Force 
technicians  served  as  subjects  for  the  study.  Each  subject  completed  12 
problems  (representative  troubleshooting  tasks).  Subjects  tested  during  the 
first  week  were  assigned  to  the  color  condition.  Subjects  tested  during  the 
second  week  were  assigned  to  the  black-and-white  condition. 

Test  Data.  It  was  determined  that  a  complex  schematic  diagram  represents 
the  most  difficult  test  of  the  capability  of  a  system  to  present  large, 
complex  graphics.  Schematics  have  many  components,  require  electronic 
symbology  and  text,  have  many  line  elements  that  are  close  together,  and 
require  the  user  to  have  access  to  all  parts  of  the  diagram,  either  at  one 
time  or  In  sequence.  A  complex  schematic  diagram  was  selected  as  the  stimulus 
material  for  use  In  evaluating  the  experimental  data  collection  techniques- 

Experimental  Tasks.  The  experimental  task  was  to  perform  simulated 
troubleshooting  tasks  on  the  receiver  transmitter  unit  (RT-728A)  of  the 
AN/APX-64(V),  Identify  Friend  or  Foe  System.  An  analyst  Identified  12 
representative  faults  that  can  occur  In  three  circuits  of  the  unit.  The  tasks 
were  judged  to  be  equal  In  difficulty  of  fault  Isolation.  The  analysts  then 
prepared  a  protocol  which,  given  that  the  subject  fault  was  present,  listed 
the  expected  result  of  each  possible  test  of  the  affected  circuits. 

Performance  Measures.  Several  performance  measures  were  used  to  evaluate 
the  subjects1  performance  under  each  condition: 

1.  TOTAL  -  The  total  effective  viewing  time  (total  time  graphics  were  on 
the  display  during  the  test  period). 

2.  TIME  -  The  mean  effective  viewing  time  used  by  the  subject  to  actually 
view  and  Interpret  the  data  on  the  screen. 

3.  TS  -  Troubleshooting  errors  (Including  Identifying  wrong  component  as 
faulty,  not  Identifying  the  fault  at  all,  taking  the  wrong  logical  path,  and 
not  being  able  to  Interpret  symbols  or  circuit  functions). 

4.  SYSTEM  -  System  errors  (errors  made  in  interacting  with  the  system, 
such  as  Incorrectly  using  NEXT,  BACK,  or  a  graphics  function). 

5.  PROCEDURAL  -  Total  number  of  Instances  requiring  unplanned 
Intervention  by  the  experimenter  to  allow  the  subject  to  continue  the  task 
(e.g.,  system  problems,  technical  data  problems). 
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All  time  measures  were  calculated  from  observations  of  videotape 
recordings  of  the  test  sessions.  Error  measures  were  calculated  from  the 
notes  of  the  experimenter,  the  observer  (an  AFWIL  representative),  and  the 
videotape  rater. 

Experimental  Procedure.  The  experiment  was  conducted  at  the  Hughes 
Aircraft  Company  facility  in  Long  Beach,  California.  A  maintenance  laboratory 
was  used.  The  computer  display  was  Installed  on  a  bench  In  the  laboratory  in 
a  manner  similar  to  that  which  would  be  used  In  an  actual  intermediate  level 
avionics  maintenance  shop.  Prior  to  the  start  of  the  experiment,  the  subject 
was  asked  to  complete  a  short  pre-test  questionnaire.  After  a  short  training 
period  on  the  use  of  the  computer  system,  the  technician  was  given  the 
troubleshooting  problem  and  Instructed  to  Identify  the  faulty  component.  The 
diagrams  presented  on  the  computer  display  were  his  only  references  for 
diagnosing  the  problems.  Since  "live"  equipment  was  not  available  to  actually 
make  required  measurements,  the  technician  was  Instructed  to  tell  the  test 
administrator  what  tests  he  would  like  to  make.  The  test  administrator  then 
gave  him  the  expected  result  of  that  test  as  given  on  the  test  protocol.  For 
example.  If  the  technician  Indicated  that  he  would  like  to  "check  the  voltage 
at  TP  5,"  the  experimenter  would  respond  (based  upon  the  test  protocol)  that 
the  output  was  "normal."  After  the  testing  was  completed,  each  subject 
completed  a  post-test  questionnaire  and  was  Interviewed  to  obtain  his  overall 
reactions,  suggestions,  and  recommendations. 

Results.  An  Analysis  of  Variance  was  performed  on  the  data  collected  for 
each  of  the  performance  measures  (total  time,  total  viewing  time,  effective 
viewing  time,  and  troubleshooting  errors),  the  system  measure,  and  the 
procedure  measure.  Analyses  were  also  made  to  determine  the  effect  of  the 
order  of  presentation  and  to  evaluate  the  comparability  of  the  problems.  The 
analyses  yielded  the  following  statistically  significant  results: 

1.  For  the  segmentation  variable,  the  mean  effective  viewing  times  (TIME 
variable)  required  by  technicians  to  perform  the  experimental  tasks  using 
pyramid  functional/schematic  diagrams  were  significantly  less  than  the  times 
required  for  technicians  to  perform  the  experimental  tasks  using  spatial 
segmentation  diagrams.  This  appeared  to  be  due  to  the  length  of  time  required 
for  successive  redrawing  of  the  diagram  when  the  scroll  feature  was  used  to 
locate  an  Item  of  interest  or  trace  a  signal  flow.  Since,  In  some  cases.  It 
was  necessary  to  redraw  the  diagram  several  times  to  locate  the  Items  of 
interest  or  to  follow  a  signal  flow,  significant  amounts  of  time  were  lost 
waiting  for  the  computer  to  display  the  correct  portion  of  diagram. 

2.  A  significant  F-value  was  obtained  for  the  PROCEDURAL  measure  for  the 
schema,  plck-and-highlight,  and  scroll  and  zoom  (baseline)  conditions  (means 
.45,  .20,  and  .16,  respectively).  The  schema  condition  required  the  least 
Intervention  by  the  experimenter. 

3.  For  the  SYSTEM  measure,  a  significant  three-way  Interaction  was  found 
for  the  color  x  segmentation  x  graphics  variables.  This  interaction  was  not 
interpretable. 

Analysis  of  the  questionnaire  and  debriefing  data  provided  the  following 
observations: 
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1.  Technicians  had  generally  sore  positive  attitudes  toward  an  automated 
technical  data  system  following  the  experiment  than  before. 

2.  Technicians  responded  negatively  to  the  two-window  schema  condition  as 
Implemented.  However,  they  generally  favored  having  an  overview  window 
available.  They  recommended  the  following  steps  to  make  the  overview 
effective: 

a.  The  overview  window  should  be  "active"  and  have  all  of  the 
graphics  capabilities  (scroll,  zoom,  and  pick-and-hlghllght). 

b.  The  Initial  presentation  of  the  overview  diagram  should  be  at  a 
scale  such  that  all  text  and  symbols  are  legible. 

c.  Other  technical  data  content  should  be  selectable  for 

presentation  In  the  second  window. 

3.  The  color-coding  condition  was  generally  rated  highly  by  the 
technicians. 

4.  The  scroll  condition  was  rated  favorably  by  the  technicians.  Four  out 
of  six  technicians  rated  It  as  easy  to  use.  The  "discontinuous"  nature  of  the 
scroll  function  was  not  seen  as  a  problem.  (As  Implemented,  when  the 
schematic  was  scrolled,  the  display  was  erased  and  the  schematic  redrawn  In 
the  new  orientation.  The  user  could  not  see  the  schematics  "move.") 

5.  The  technicians  rated  the  cursor  control  (puck)  as  acceptable.  It  was 
suggested  that  the  function  keys  be  Included  on  the  puck  Itself. 

6.  The  menu  used  to  select  the  graphics  functions  was  seen  as  acceptable, 
but  somewhat  slow. 

7.  The  technicians  agreed  that  the  response  time  (time  from  request  until 
the  frame  was  displayed)  was  too  slow.  Response  times  of  1  to  2  seconds  were 
recommended. 

Recommendations.  Based  upon  the  experimental  data  and  the  results  of  the 
questionnaire  3ata  and  post-test  Interviews,  Hughes  personnel  made  the 
following  recommendations  (Hughes  Aircraft  Company,  1985a,  p.  35): 

1.  The  most  critical  finding  of  the  study  is  that  technicians  are  able  to 
troubleshoot  more  effectively  when  using  a  series  of  hierarchical  (pyramid) 
functional  diagrams  presented  on  a  computer  than  when  using  one  large 
schematic  presented  on  a  computer  display  and  viewed  using  scroll  and  zoom 
functions.  Therefore,  the  CMAS  specifications  should  provide  for  the  use  of 
hierarchical  or  pyramid  diagrams  for  the  presentation  of  complex  diagrams.  In 
addition,  the  system  should  provide  a  rapid  and  simple  method  for  selecting 
and  displaying  alternative  diagrams  from  the  pyramid  set. 

2.  Statistically  significant  differences  were  not  found  for  the  Color, 
Resolution,  and  Graphic  Interaction  conditions.  Therefore,  these  variables 


could  not  be  recommended  for  inclusion  in  the  CMAS  specification  on  the  basis 
of  this  study.  The  subjects  did  not  demonstrate  significantly  improved 
performance  with  the  high-resolution  screen  nor  did  the  presence  of  color 
reliably  improve  the  performance  of  the  technicians. 

3.  The  hardware  used  for  the  study  resulted  in  an  unacceptably  slow 
response  time.  A  significantly  improved  response  time  is  necessary  for  a 
production  CHAS  system. 

The  minimum  capabilities  for  the  prototype  should  include  a  black-and- 
white,  512  x  512  graphics  display  with  zoom  and  scroll  capability.  The 
graphics  data  should  be  prepared  in  a  pyramid  format  such  that  each  subdiagram 
would  be  presented  in  one  screen  with  lines,  symbols,  and  text  legible  at 
first  presentation,  and  with  functionally  grouped  graphic  elements  available 
for  highlighting. 

Design  Study  3 

Purpose.  The  purpose  of  Design  Study  3  was  to  evaluate  the  effects  of: 
(a)  integration  and  nonintegration  of  text  and  graphics,  (b)  the  level  of 
detail  of  graphics,  (c)  the  use  of  color-coding,  and  (d)  resolution  of  the 
display  on  the  performance  of  selected  nontroubleshooting,  procedural  tasks. 

Variables.  Four  variables  were  examined  In  the  study. 

1.  Integration  Method.  Two  methods  were  evaluated: 

a.  Fully  Integrated  -  Text  was  presented  as  an  overlay  to  the 
graphic  image  such  that  the  text  appeared  In  close  proximity  to  the  referenced 
components.  A  callout  arrow  pointed  from  the  text  to  the  referenced  com¬ 
ponent.  The  sequence  of  steps  was  indicated  by  numbering  the  steps.  The  text 
was  subject  to  the  same  Interaction  functions  as  the  graphic.  The  concept  is 
Illustrated  In  Figure  33.  (A  sample  frame  from  the  study  was  not  available 
for  Inclusion  in  this  report.) 

b.  Nonintegrated  -  The  text  and  graphics  were  Independent.  The  text 
was  presented  In  a  dedicated  text  window  on  the  left  side  of  the  display  and 
subject  only  to  text  manipulation  functions.  The  graphic  was  presented  in  a 
dedicated  window  on  the  right  side  of  the  display.  Callout  arrows  on  the 
graphic  were  keyed  to  the  text.  The  format  was  similar  to  that  shown  in 
Figure  30. 

2.  Color-Coding.  Two  color  conditions  were  used: 

a.  Color  Coded  -  Coding  was  based  on  the  function  of  the  component 
(e.g.,  connectors  were  presented  in  one  color;  screws  were  presented  In 
another  color). 

b.  Black  and  White  -  All  components  were  shown  in  black  and  white. 

3.  Detail  Level.  Two  levels  of  detail  for  presentation  of  graphics  were 
evaluated: 
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lure  33.  Exanple  of  Integrated  Text  and  Diagnostic  Concept 


a.  High  Detail  -  The  graphics  contained  all  of  the  detail  provided 
in  the  original  paper-based  technical  order  graphics  with  the  exception  of 
shading,  screw  threading,  connector  detail,  and  textural  detail.  All  parts 
were  shown  whether  referenced  or  not. 

b.  Low  Detail  -  The  same  basic  graphic  used  for  the  high  detail 
condition  was  used,  with  the  exception  that  only  the  parts  referenced  in  the 
text  were  shown.  Nonreferenced  parts  which  were  part  of  the  structural  frame 
were  retained  but  reduced  to  an  outline. 

Experimental  Design.  A  split-plot  factorial  analysis  of  variance  model 
with  one  between-subjects  variable  (resolution)  and  three  repeated  measures 
variables  (color,  integration  method,  and  level  of  detail)  was  used.  Each 
subject  received  eight  experimental  conditions.  Random  assignment  was  made  to 
the  resolution  variable.  Six  Air  Force  technicians  (three  high-experience  and 
three  low-experience)  served  as  subjects  for  the  study. 

Experimental  Tasks.  Four  tasks  for  the  maintenance  of  the  RT-728A  of  the 
AN/APx-64  Identify  Friend  or  Foe  System  were  used  as  the  experimental  tasks: 

1.  Remove  RF  Module 

2.  Install  RF  Module 

3.  Remove  Diode  from  the  RF  Module 

4.  Install  Diode  in  the  RF  Module 

Experimental  Procedure.  The  experimental  procedure  was  the  same  as  that 
used  in  Design  Study  2.  The  pre-  and  post-test  questionnaires  used  in  Design 
Study  2  were  administered.  In  addition,  a  new  questionnaire  relating  to  the 
specific  features  evaluated  in  this  study  was  administered  following  the 
testing  period.  The  experimental  tasks  were  performed  in  the  maintenance 
laboratory  used  for  Design  Study  2.  The  workbench  and  computer  display  were 
set  up  in  a  similar  manner.  The  subject's  performance  was  observed  by  the 
experimenter,  who  took  notes  on  performance  errors,  deviations  from  experi¬ 
mental  procedures,  system  problems,  etc.  The  sessions  were  videotaped. 

Performance  Measures.  The  following  performance  measures  were  used: 

1.  TOTAL  -  The  total  effective  viewing  time  (total  time  graphics  were  on 
the  display  during  the  test  period). 

2.  TIME  -  The  mean  effective  viewing  time  used  by  the  subject  to  actually 
view  and  interpret  the  data  on  the  screen. 

3.  MAINTENANCE  ERRORS  -  The  number  of  errors  observed  by  the  experimenter 
during  the  test.  These  included  performing  the  wrong  action,  using  the  wrong 
tool,  performing  actions  in  the  wrong  sequence,  and  orienting  parts  in  the 
wrong  way. 

4.  SYSTEM  -  System  errors  (errors  made  in  interacting  with  the  system, 
such  as  incorrectly  using  the  computer  commands  NEXT  or  BACK  or  a  graphics 
function). 


5.  PROCEDURAL  -  Total  number  of  instances  requiring  unplanned  interven¬ 
tion  by  the  experimenter  to  allow  the  subject  to  continue  the  task  (e.g., 
system  problems  and  technical  data  problems). 

In  addition,  the  initial  level  of  detail  selected  by  each  subject  was 
recorded  for  each  trial.  The  number  of  track  changes  was  also  recorded.  All 
time  measures  were  calculated  from  observations  of  videotape  recordings  of  the 
test  sessions.  Error  measures  were  calculated  from  the  notes  of  the 
experimenter  and  the  videotape  rater. 

Results.  Analysis  of  variance  techniques  applied  to  the  performance 
measures  yielded  one  statistically  significant  main  effect.  A  statistically 
significant  F-value  (j3  <  .05)  was  found  for  the  TIME  measure  for  the  color 
condition.  The  mean  effective  time  required  to  perform  the  experimental  tasks 
was  104.1  seconds  under  the  color  condition  and  117.8  seconds  under  the  black- 
and-white  condition. 

Statistically  significant  main  effects  were  not  found  for  the  resolution, 
detail  level,  and  integration  method  variables.  However,  the  authors  (Hughes 
Aircraft  Company,  1985a)  noted  that 

...the  means  for  the  effects  suggest  that  subjects  performed 
faster  whenever  there  was  less  visual  information  on  the  screen. 

Low  resolution  (103.7  seconds)  was  less  than  high  resolution 
(118.1),  and  low  detail  (99.5)  was  less  than  high  detail  (124.5). 

The  fully  integrated  mean  (98.9)  was  less  than  the  non-integrated 
mean  (125.0).  However,  further  research  is  required  to  determine 
whether  actual  effects  are  associated  with  these  observed  values. 

(P.  49) 

No  statistically  significant  differences  were  observed  for  the  MAINTENANCE 
ERRORS  measure  and  the  TRACK  CHANGES  measures. 

The  results  of  the  questionnaires  were  generally  consistent  with  the 
findings  from  Design  Study  2.  Four  of  the  six  subjects  were  interviewed 
following  the  test.  Significant  observations  from  the  interviews  included: 

1.  All  subjects  felt  that  the  most  detailed  level  (Track  3)  was 
patronizingly  simple.  The  less  detailed  Track  2  was  preferred.  It  was 
recommended  that  the  complexity  of  tasks  be  considered  in  determining  whether 
two  or  three  tracks  of  technical  information  are  required. 

2.  All  subjects  were  able  to  use  the  zoom  with  little,  if  any, 
difficulty.  The  zoom  capability  was  used  primarily  to  enlarge  the  step  text 
for  the  integrated  test  condition.  The  zoom  capability  was  recommended  for 
the  prototype  system. 

3.  The  scroll  feature  was  not  used  by  any  of  the  subjects. 

4.  The  subjects  were  split  in  their  opinions  on  the  value  of  color-coded 
graphics.  Two  subjects  preferred  the  color  condition.  The  other  two  subjects 
did  not  feel  that  there  was  any  difference  in  the  readability  of  the  color  and 
monochrome  graphics. 


5.  The  subjects  stated  that  they  were  unaware  that  some  trials  were 
performed  under  high-resolution  conditions  while  others  were  conducted  under 
low-resolution  conditions.  There  did  not  appear  to  be  any  significant 
difference  in  the  usability  of  the  high-  and  low-resolution  presentations. 
Readability  seemed  to  be  more  a  function  of  the  absolute  size  of  the  data  on 
the  screen  than  image  resolution.  It  was  recommended  that  the  lower- 
resolution  display  (512  x  512  pixels)  be  used  for  the  prototype. 

6.  All  subjects  found  the  low  detail  graphics  to  be  more  usable  than  the 
more  detailed  graphics.  Low  detail  graphics  were  recommended  for  the 
prototype  system. 

7.  All  subjects  complained  that  the  time  required  to  download  graphics  to 
the  terminal  was  too  long.  It  was  recommended  that  the  prototype  system 
provide  for  much  faster  download  times  (although  no  specific  recommendations 
were  given  by  the  subjects). 

8.  The  subjects  generally  agreed  that  the  experimental  tasks  were  too 
easy  to  adequately  test  the  system's  delivery  features.  It  was  recommended 
that  more  challenging  tasks  be  used  for  future  tests. 

Discussion  and  Recommendations.  The  researchers  made  the  following 
recommendations  (Hughes  Aircraft  Company,  1985a,  p.  50)  based  upon  the 
experimental  data,  questionnaire  responses,  and  post-test  interview  comments: 

1.  The  most  significant  finding  of  the  study  was  the  effect  of  color  on 
the  mean  effective  performance  time.  This  finding  was  interpreted  to  mean 
that  functional  color-coding  of  equipment  parts  on  isometric  illustrations 
facilitates  performance  of  procedural  maintenance  tasks.  The  functional 
specifications  for  the  CMAS  prototype  should  include  the  requirement  for 
encoding  of  a  maximum  of  16  discriminable  colors  and  for  the  functional  coding 
of  isometric  illustrations  such  that  equipment  parts  may  be  identified  as 
functional  units. 

2.  Further  research  is  needed  to  assess  the  effects  of:  (a)  integration 
of  text  and  graphics  in  procedural  instructions,  (b)  the  effects  of  detail 
level  of  graphics,  and  (c)  resolution.  The  type  and  complexity  of  the 
procedures  and  graphics  used  in  the  study  did  not  place  stringent  enough 
requirements  on  the  system  to  adequately  evaluate  these  variables. 

3.  There  were  negative  ratings  of  the  CMAS  as  implemented  in  the  design 
studies  due  to  hardware  limitations  which  resulted  in  very  slow  response 
times.  This  finding  was  consistent  for  all  three  studies.  Minimum  response 
times  and  maximum  baud  rates  between  host  computer  and  graphics  processor 
should  be  required  for  any  CMAS  configuration  in  the  subsequent  phases  of  the 
program. 

4.  The  experimental  procedures  used  in  Design  Studies  2"and  3  were  very 
complex.  A  number  of  procedural  irregularities  occurred.  Thus,  caution 
should  be  used  in  generalizing  from  the  results  of  the  studies,  as  discussed 
below. 


Comments  on  Design  Studies 


The  design  studies  provided  useful  information  for  the  design  of  the  CMAS 
prototype  and  provided  valuable  experience  for  the  system  developers.  However, 


they  did  not  provide  the  type  of  definitive  design  information  that  had  been 
desired.  A  number  of  weaknesses  in  the  design  and  implementation  of  the 
studies  make  it  necessary  to  use  caution  in  generalizing  from  the  findings. 
The  following  paragraphs  illustrate  the  problems/weaknesses  of  the  studies. 

1.  Experimental  Tasks.  The  experimental  tasks  selected  for  Design 
Studies  1  and  3  were  much  too  simple  to  adequately  test  the  variables  being 
evaluated.  The  tasks  were  simple  remove  and  replace  tasks.  An  experienced 
technician  (and  even  a  novice  with  a  good  mechanical  aptitude)  would  have  been 
able  to  successfully  perform  the  tasks  without  using  technical  data.  The 
tasks  were  very  simple,  requiring  only  a  few  easy  steps  (as  evidenced  by  the 
fact  that  the  mean  time  to  accomplish  each  task  was  less  than  2  minutes).  As 
a  result,  the  ability  of  the  system  to  provide  the  technician  with  instruc¬ 
tions  on  how  to  do  the  task  had  little  impact  on  performance.  Similarly,  the 
ability  of  graphics  to  aid  in  locating  components  on  the  equipment  was  not 
truly  tested.  There  were  no  hidden,  unique,  or  hard-to-find  components  that 
the  technician  needed  assistance  in  finding.  Further,  the  components 
illustrated  were  relatively  simple  and  did  not  present  the  number  of  parts  or 
the  complexity  often  encountered  in  technical  data. 

2.  Hardware/Sof tware  L i mi  tat ions.  The  limited  capabilities  of  the 
hardware  and  software  available  pTaced  significant  constraints  on  the 
studies.  The  system  was  able  to  provide  all  of  the  required  functions. 
However,  it  was  not  able  to  provide  the  response  times  required  for  effective 
presentation  of  technical  data.  This  was  especially  a  problem  for  complex 
graphics  such  as  those  used  in  Design  Study  2.  Several  minutes  were  required 
to  draw  some  of  the  graphics.  Similar  delays  were  experienced  in  zooming  and 
scrolling  the  display  since  these  actions  required  redrawing.  This  forced  the 
use  of  an  approach  to  zooming  and  scrolling  which  involved  erasing  the  screen 
and  redrawing  it  several  seconds  later.  The  delay  increased  the  risk  of  the 
user  losing  his  place.  The  impact  of  this  limitation  on  the  results  of  the 
spatial  segmentation  versus  functional  segmentation  schematic  diagrams  com¬ 
parisons  in  Design  Study  2  is  not  known.  Similarly,  the  impact  of  response 
speed  on  the  acceptance  of  the  automated  technical  data  system  concept  by  the 
technicians  is  not  known. 

3.  Sample  Size.  The  sample  size  for  the  studies  was  too  small.  Only  six 
subjects  were  used  for  each  study  (availability  of  subjects  was  limited  by 
travel  funds).  With  such  a  small  sample,  a  relatively  large  difference  in 
means  is  required  to  demonstrate  a  statistically  significant  difference  in 
performance.  As  a  result,  the  study  may  have  failed  to  detect  meaningful 
differences  in  the  effectiveness  of  some  of  the  variables  studied.  Hindsight 
suggests  that  more  meaningful  results  might  have  been  obtained  had  the  studies 
been  combined  and  all  18  subjects  experienced  each  condition. 

Perhaps  the  most  significant  finding  of  the  design  studies  was  that,  in 
spite  of  the  problems,  the  concept  of  automated  technical  data  systems  was 
well  received  by  the  technicians.  They  were  willing  to  overlook  the 
weaknesses  of  the  test  system  because  they  could  see  the  potential  benefits  of 
an  automated  system  (with  the  identified  weaknesses  corrected).  Other 
significant  findings  include:  (a)  the  apparent  advantage  of  the  "pyramid" 
approach  for  presenting  schematic  diagrams,  (b)  the  apparent  advantages  of 
color-coding  for  schematic  diagrams  and  possibly  for  functional  coding  of 


equipment  drawings,  (c)  the  value  of  keeping  presentations  simple  by 
eliminating  unnecessary  detail,  and  (d)  the  apparent  lack  of  any  need  for  a 
resolution  level  higher  than  512  x  512  pixels  (on  the  11.6  x  15.1  inch 
display).  Further  testing  of  these  features  is  essential  before  firm 
requirements  can  be  established.  The  tests  should  evaluate  the  features  in  a 
wider  variety  of  applications  and  more  challenging  situations. 


Man/Machine  Interface  Design 

Based  upon  the  results  of  the  literature  surveys  and  the  design  studies, 
the  contractors  developed  a  proposed  MMI  design  for  the  CMAS.  The  proposed 
design  represents  what  was  felt  to  be  necessary  for  the  most  effective 
system.  All  of  the  features  specified  were  not  included  in  the  prototype 
system  for  a  number  of  reasons.  The  basic  MMI  requirements  (Hughes  Aircraft 
Company,  1985b,  pp.  3-49  through  3-52)  are  described  in  the  following 
paragraphs: 

Graphics  Display  Mode  Function.  The  graphics  files  must  be  displayed  in 
various  ways  through  the  selection  of  the  display  functions  listed  below. 
Each  function  must  be  associated  with  a  data  table  of  options.  All  options 
must  be  selectable.  (The  purpose  of  the  graphics  display  mode  is  to  provide 
several  alternatives  to  the  user  for  test  and  comparison  of  the  effectiveness 
of  graphics  presentations  under  a  variety  of  conditions.) 

1.  Window  Definition  -  The  user  must  be  able  to  define  portions  of  the 
display  screen  (e.'gJ",  1(524  x  1024  pixel  array  subdivided  into  rectangular 
areas)  for  display  of  portions  of  the  graphics  data  base.  Window  size  and 
placement  must  be  definable,  not  fixed. 

2.  Viewports  -  The  user  must  be  able  to  define  the  portion  of  the 
graphics  data  base  or  file  to  be  displayed  in  a  given  window. 

3.  Graphics  Files  -  The  user  must  be  able  to  define  and  display  multiple 
portions' of  two  or  more  files  in  multiple  windows. 

Resolution.  The  user  must  be  able  to  select  either  low  or  high  (512  x  512 
or  1 024  x  1024  pixels)  resolution  for  the  full  screen.  (Resolution  was 
controlled  by  experimenters,  not  users,  during  the  studies.) 

Character  Size.  The  user  must  be  able  to  select  either  a  character  size 
appropriate  for  a  close  viewing  distance  (2  feet)  or  character  size  suitable 
for  a  far  viewing  distance  (6  feet)  (character  heights  of  .13  and  .21  inch, 
respectively)  for  standard  text  elements  of  the  graphic.  (Not  implemented.) 

Interactive  Processes.  The  user  must  be  able  to  input  commands  at  the 
keyboard  to: 

1.  Enter  text  to  an  overlay  plane  to  annotate  portions  of  the  graphic. 
The  text  file  must  be  associated  with  the  user's  identification  code,  and  must 
be  retrievable  by  the  user  for  display  at  a  later  session.  (Not  implemented.) 

2.  Pick  graphic  segments  by  indicating  position  with  cursor  or  coordinate 
commands. 
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3.  Scroll  and  zoom  by  moving  a  viewport  over  the  graphics  data  base. 
Relative  movement  of  a  window  must  appear  to  the  user  to  be  a  window  moving 
over  a  fixed  graphic  in  response  to  user  inputs  (e.g.,  cursor  command  to  go 
left  moves  the  window  left  relative  to  the  graphic  which  appears  fixed). 

Scroll  must  appear  as  a  movement  of  the  window  left/right  and  up/down  at  a 
rate  of  approximately  10  frames  per  second  and  a  displacement  of  .25  window 
widths  per  frame.  (Not  implemented  to  simulate  continuous  scroll  due  to  slow 
refresh  rate.)  Zoom  must  appear  as  an  increase  or  decrease  of  image  size 
within  the  displayed  window  at  a  rate  of  approximately  10  frames  per  second 
and  a  change  of  .25  window  widths  per  frame.  (Not  implemented  to  simulate 
continuous  zoom  due  to  slow  refresh  rate.) 


Fabrication  of  the  CMAS  Prototype 

The  next  task  was  to  fabricate  the  prototype  CMAS  for  use  in  the  field 
test.  The  approach  was  to  take  the  hardware  used  in  the  design  studies,  the 
NTIPS  software,  and  the  special  software  developed  for  the  design  studies,  and 
to  adapt  and  expand  them  to  meet  the  MMI  requirements— to  the  extent  that 
resources,  technical  capabilities,  and  time  constraints  would  permit. 

The  hardware  configuration  is  presented  diagrammatically  in  Figure  34. 
The  system  was  composed  of  the  following  hardware: 

-  M0DC0MP  7840  Computer  System  (host  system)  with  a  MAX  IV  resident 
operating  system,  hard  disk  drives,  and  tape  drive. 

-  CONRAC  Model  C3919NPL  High  Resolution  Color  Graphics  Display 

(1024  x  1024  pixels). 

-  Rastertech  Model  0ne/80  (modified)  graphics  processor. 

-  Microframe  translator  and  keyboard  processor. 

-  Detachable  keyboard. 

-  Mouse. 

-  Parallel  Interface  between  MODCOMP  and  Rastertech. 
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The  following  software  was  incorporated  into  the  system: 

-  Existing  NTIPS  Electronic  Display  Nodule  (EDM  -  hosted  on  MODCOMP), 
modified  to  satisfy  CMAS  Display  Program  software  specification  requirements. 

-  Color  graphics  display  module  (hosted  on  Microframe)  developed  to 
control  interactive  display  of  512-  or  1024-pixel  resolution  graphics  and 
advanced  features. 

-  Parallel  interface  software  to  increase  speed  of  graphics  data  transfer. 

-  Fault  Isolation  by  Nodal  Dependency  (Find)  software  modified  to  operate 
with  the  EDM. 

-  SIMPLER  software  package  to  control  user  interaction  processes. 

-  Other  special  purpose  software  such  as  drivers  to  interface  hardware, 
and  special  graphics  manipulation  techniques. 

A  number  of  problems  were  encountered  in  fabricating  the  prototype.  Some 
of  the  problems  prevented  satisfying  all  of  the  MMI  requirements  specified 
above.  Some  of  the  problems  were: 

1.  Response  time  requirements  for  data  containing  graphics  were  not  met. 
The  problem  was  due  to  several  factors,  including  the  fact  that  several  time- 
consuming  conversions  of  the  graphics  between  the  host  computer  and  the 
display  were  required. 

2.  A  continuous  Scroll  and  Zoom  feature  was  not  possible.  Continuous 
scrolling  and  zooming  would  have  required  extensive  modifications  to  the  CMAS 
graphics  display  system  and  separation  from  the  MODCOMP  host  system. 


Technical  Data  Development 

The  next  step  in  the  program  was  to  develop  a  set  of  technical  data  for  a 
representative  testbed  system  for  use  in  evaluating  the  CMAS.  Technical  data 
were  developed  for  portions  of  the  Receiver/Transmitter  unit  (RT-728A)  of  the 
AN/APX-64  Identify  Friend/Foe  System.  The  AN/APX-64  is  used  on  several  Air 
Force  aircraft  including  the  KC-135.  The  procedures  used,  problems  en¬ 
countered,  and  recommendations  developed  from  this  task  are  summarized  below. 


Front-End  Analysis 

A  detailed  front-end  analysis  (FEA)  was  conducted  to  ensure  that  the  data 
developed  were  complete,  technically  accurate,  and  written  at  a  level  of 
detail  appropriate  for  the  intended  users.  The  FEA  included  the  following 
analyses: 


1.  Detailed  Task  Analysis.  A  systematic  analysis  was  undertaken  to 
identify  each  task  performed  on  the  subject  equipment  to  ensure  that  all  tasks 
were  covered;  to  identify  the  information  that  the  technician  requires  to  do 
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each  task;  to  establish  a  list  of  distinctive  and  convenient  common  names  for 
use  throughout  the  data;  to  identify  required  supplies*  parts,  support 
equipment,  etc.;  and  to  develop  effective  procedures  to  accomplish  each  task. 


2.  Detailed  User  Analysis.  A  thorough  analysis  of  the  anticipated 
capabilities  of  the  users  was  made  to  develop  a  description  of  the  user.  The 
description  was  provided  to  the  technical  data  writer  as  a  guide  to  ensure 
that  the  data  were  written  at  an  appropriate  level  of  detail.  Since  the  data 
were  to  be  written  in  three  levels  of  detail ,  it  was  necessary  to  consider  the 
capabilities  of  three  categories  of  technicians  (novice,  experienced,  and 
highly  experienced)  in  the  analysis. 

The  results  of  the  above  analyses  were  then  used  to  develop  guidelines  for 
determining  the  amount  of  detail  of  the  instructions  written  for  each  track. 


Technical  Data  Development 

Following  the  FEA,  the  following  types  of  technical  data  to  be  used  in  the 
field  evaluation  were  developed: 

1.  Corrective  Maintenance.  The  corrective  maintenance  data  were 
developed  in  a  modified  Job  Guide  format  in  three  tracks.  All  tracks  included 
illustrations  (isometric  graphics),  warnings,  cautions,  notes,  input 
conditions,  tolerance  values,  and  data  access  options.  Track  1  provided 
general  descriptions  of  the  task.  Track  2  included  step-by-step  instructions. 
Track  3  included  step-by-step  instructions  plus  descriptions  of  special 
techniques,  identification  of  tools  for  each  step,  and  other  information 
required  to  aid  an  inexperienced  technician  in  completing  the  task.  The 
graphics  were  interactive.  The  user  was  able  to  scroll,  zoom,  and  highlight 
areas  of  the  graphic. 

2.  Troubleshooting  Procedures.  The  FIND  system  was  used  to  develop  and 
present  troubleshooting  procedures.  Schematic  diagrams,  wiring  diagrams, 
voltage  tables,  and  other  data  were  analyzed  to  develop  the  parameter  data 
needed  for  input  to  the  FIND  model.  The  FIND  model  uses  component,  signal, 
and  dependency  information  to  generate  troubleshooting  procedures  by 
computer.  The  FIND  procedures  were  presented  in  three  tracks.  Track  1 
included  symptom  summary  tables  from  the  FIND  model.  Track  2  Included  test 
requirements  identified  by  the  FIND  model.  Track  3  included  test  requirements 
identified  by  the  FIND  model,  along  with  illustrations  for  test  point  location 
and  procedures  for  performing  the  tests.  In  addition  to  the  track  data, 
schematic  diagrams  were  developed.  The  schematics  were  developed  in  the 
spatial  segmentation  format  used  for  Design  Study  2.  Color-coding  was  used  to 
identify  functionally  related  components.  The  schematics  were  available  to 
the  user  as  pool  data. 

3.  Illustrated  Parts  Breakdown  Data.  The  IPS  data  were  developed  with 
little  additional  analysis.  The  primary  task  was  to  organize  and  format  the 
data  for  input  into  the  CMAS.  The  IPB  data  were  presented  in  the  form  of 
complex  isometric  graphics.  The  graphics  provided  sufficient  detail  to  allow 
the  user  to  recognize  individual  parts.  The  graphics  also  served  as  a  graphic 
index,  in  that  the  user  was  able  to  enter  a  code  or  callout  number  from  the 
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graphic  and  retrieve  the  pertinent  information  relevant  to  the  component  of 
interest.  The  user  was  able  to  access  parts  information  in  the  following 
ways:  (a)  He  could  select  the  appropriate  graphic  from  a  menu  and  then  use 
the  graphic  as  a  menu  to  select  Information  on  specific  components;  (b)  he 
could  select  a  table  of  part  numbers  or  reference  designators  and  call  up  a 
graphic  showing  the  item  of  interest;  or  (c)  he  could  use  a  direct  access  mode 
to  call  up  a  graphic  and  parts  information  by  directly  entering  the  part 
number  or  reference  designator. 

It  should  be  noted  that,  due  to  cost  considerations,  technical  data 
produced  for  the  field  test  did  not  Incorporate  many  of  the  design  criteria 
identified  in  the  system  design  developed  earlier  in  the  effort.  Points  of 
deviation  Included: 

1.  Three- track  data  were  available  only  for  materials  covered  in  Section 
9  of  the  TO  (Intermediate  Frequency  Amplifier).  The  checkout  procedure  and 
pool  information  were  presented  with  only  one  track.  Also,  it  proved  to  be 
very  difficult  to  develop  three  distinctively  different  tracks  for  the  testbed 
data.  In  most  cases,  it  was  difficult  to  develop  a  more  enriched  Instruction 
for  Track  3  than  that  provided  for  Track  2.  For  example,  it  Is  difficult  to 
enrich  an  Instruction  to  "Turn  ON/OFF  switch  (1)  to  ON."  The  convention 
adopted  to  provide  enriched  Track  3  data  was  to  specify  the  tool  to  be  used. 
This  information  frequently  was  obvious  and  unnecessary.  For  example,  even 
the  technician  with  no  experience  does  not  need  to  be  told  to  use  a 
screwdriver  to  remove  a  screw.  Another  example  from  Track  3  data  was  the 
instruction  "Using  hands,  turn  ON/OFF  switch  (1)  to  ON."  Such  superfluous 
instructions  were  considered  to  be  a  joke  by  the  technicians. 

2.  The  functional  segmentation  approach  identified  as  superior  in  Design 
Study  2  was  not  used.  The  spatial  segmentation  method  with  functional 
color-coding  was  used  in  its  place. 

3.  The  locator  illustrations  used  with  procedural  tasks  contained  every 
callout  used  for  any  step  in  the  data  base.  The  earlier  guidelines  had 
indicated  that  only  those  callouts  referenced  on  that  specific  frame  should  be 
shown. 

4.  Pool  information  (other  than  schematics  which  were  color-coded)  was 
not  restructured  for  presentation  on  the  CMAS.  Information  such  as  theory  of 
operation,  system  description,  test  equipment,  and  special  tools  listings, 
etc.  were  copied  verbatim  from  the  TO. 


Validation  of  Technical  Data 

Validation  of  the  technical' data  was  accomplished  as  described  below. 

1.  Validation  of  Corrective  Maintenance  Data.  Validation  of  the 
corrective  maintenance  procedures  was  accomplished  at  the  Hughes  Aircraft 
Company,  using  equipment  provided  by  AFHRL.  Validation  consisted  of 
successful  completion  of  100%  of  the  procedures  to  be  used  in  the  field  test. 

2.  Validation  of  the  Troubleshooting  Data.  The  troubleshooting  pro¬ 
cedures  were  validated  at  March  AF6,  California.  Validation  was  accomplished 


by  completing  all  troubleshooting  procedures  using  an  active  test  bench  In  the 
Intermediate  level  shop  at  March  AFB.  The  FIND  system  was  validated  by  using 
It  to  successfully  fault-lsolate  a  sample  of  faults  at  the  system  (line 
replaceable  unit)  level  (80%  of  the  faults  were  hlgh-frequency-of-occurrence 
faults  and  20%  of  the  faults  were  selected  at  random).  The  validation  was 
accomplished  using  a  black-and-white  terminal  connected  by  modem  to  the  host 
computer  at  the  Hughes  facility. 

3.  Illustrated  Farts  Breakdown  Data.  The  IPB  data  were  validated  at  the 
Hughes  facility  by  Hughes'  subject-matter  experts.  Validation  consisted  of 
comparing  the  electronic  data  with  the  original  TO. 

4.  Pool  Data.  The  pool  data,  consisting  of  schematics  and  theory  of 
operation  data,  were  validated  at  the  Hughes  facility  by  Hughes'  subject- 
matter  experts.  Validation  was  accomplished  by  comparison  of  the  data  with 
the  TO. 

5.  System  Validation.  The  validation  of  the  integration  of  the  data  and 
the  CMAS  was  conducted  at  the  Hughes  facility  by  Hughes  Aircraft  Company, 
Rockwell  International,  and  AFHRL  personnel.  The  validation  consisted  of 
successfully  accessing  a  sample  data  base  In  accordance  with  realistic 
maintenance  scenarios  and  successfully  locating  the  Information  required  to 
perform  the  maintenance  tasks  specified  In  the  scenarios. 

Conclusions  from  the  Data  Development  Task.  Experience  In  the  development 
of  technical  data  for  £Ei  cMAs  led  the  contractors  to  the  following 
conclusions  and  observations: 

1.  Three-Track  Procedural  Data.  Procedural  data  cannot  be  developed  In 
three  tracks  for  all  types  of  data  and  applications.  The  establishment  of 
tracks  and  level-of-detall  standards  must  be  based  upon  the  nature  and 
complexity  of  the  equipment  being  described.  Task  complexity,  more  than  user 
characteristics,  drives  the  need  for  multiple-track  data  and  is  the  basis  for 
determining  the  number  of  tracks  required  for  a  particular  application. 
Electronic  systems  tend  to  be  very  complex  to  troubleshoot  and  simple  to 
repair.  Thus,  for  these  systems,  multitrack  presentation  may  be  required  for 
troubleshooting.  Procedural  repair  tasks  do  not  seem  to  require  multiple 
levels  of  detail.  Mechanical  systems,  however,  present  a  different  set  of 
documentation  problems.  The  fault  Isolation  and  repair  processes  are  not 
neatly  separated,  but  are  closely  related.  In  addition,  mechanical  systems 
are  more  complex  to  disassemble,  inspect,  and  repair.  Multitrack  data  are 
appropriate  and  necessary  for  such  systems.  Nonetheless,  It  Is  hardware 
complexity  rather  than  user  performance  potential  that  is  the  primary  driver 
in  determining  the  need  for  a  multitrack  approach. 

2.  FEA  Requirements.  All  data  base  planning  was  found  to  depend  com¬ 
pletely  on  a  comprehensive  FEA  which  defines.  In  detail,  all  data  require¬ 
ments.  This  provides  a  firm  basis  for  planning  and  managing  the  data 
development  process.  In  addition.  It  enables  the  procuring  activity  to  more 
effectively  monitor  the  contractor's  data  development  process.  An  FEA  should 
be  mandatory  for  the  procurement  of  technical  data  for  automated  systems  such 
as  the  CMAS. 

3.  Content  Specification.  It  1*:  essential  that  existing  detail  or 
content  specifications  be  modified,  or  new  specifications  be  generated,  to 


provide  specific  requirements  for  automated  technical  data  for  use  at  the 
various  levels  of  maintenance. 


Field  Test 

The  field  test  was  conducted  at  Offutt  AFB,  Nebraska,  5  November  1984 
through  11  January  1985.  Facilities  of  the  55th  Reconnaissance  Wing  were  used 
for  the  test.  A  number  of  problems  were  encountered  in  conducting  the  test 
which  prevented  accomplishing  the  field  test  as  originally  planned.  A 
substitute  evaluation  approach  was  developed  and  implemented  by  AFHRL 
personnel.  The  original  field  test  plan,  the  work  accomplished  under  that 
plan,  the  substitute  plan,  and  findings  of  the  test  are  discussed  below. 


Field  Test  Plan 


The  original  test  plan  (Hughes  Aircraft  Company)  provided  for  an 
operational  field  test  in  which  the  CMAS  system  would  be  placed  in  an 
intermediate  level  shop  responsible  for  maintaining  the  testbed  system.  The 
CMAS  was  to  be  made  available  for  use  by  the  technicians  for  their  normal, 
day-to-day  maintenance  of  the  system.  It  was  to  be  used  to  perform 
troubleshooting  and  corrective  maintenance  tasks  as  the  requirements 
occurred.  Data  were  to  be  recorded  by  the  technicians  on  the  use  of  the 
system  and  problems  encountered  in  using  it.  Pretest  questionnaires  were  to 
be  administered  to  all  personnel  in  the  Radar  Shop,  the  technicians  were  to 
use  the  system  for  a  period  of  time,  data  were  to  be  logged  on  the  use  of  the 
system,  and  questionnaires  and  interviews  were  to  be  administered  at  the  end 
of  the  test  period.  The  plan  did  not  provide  for  a  more  formal  experimental 
test  using  performance  tests.  Hughes  personnel  had  the  primary  responsibility 
for  data  collection  under  this  plan. 

As  a  result  of  problems  encountered  in  implementing  the  original  test 
plan,  it  was  determined  that  an  adequate  evaluation  could  not  be  achieved  by 
depending  upon  use  of  the  system  for  actual  maintenance  tasks.  A  substitute 
plan  was  developed  and  Implemented  by  AFHRL.  This  plan  provided  for  the  use 
of  ”set  up"  problems  to  provide  a  means  of  collecting  limited  performance  data 
and  to  ensure  that  the  technicians  had  an  adequate  opportunity  to  experience 
using  the  system.  Twelve  technicians  (six  high-experience  and  six 
low-experience)  served  as  subjects.  Each  subject  completed  four  problems: 
two  using  the  CMAS  and  two  using  the  paper  TO.  The  problems  included  two 
checkout  tasks,  and  two  remove  and  replace  tasks. 

In  effect,  two  concurrent  evaluations  were  conducted.  The  initial  plan 
was  implemented  by  Hughes  to  the  extent  feasible.  The  Second  plan  was 
implemented  by  AFHRL.  The  results  of  each  evaluation  are  summarized  in  the 
following  paragraphs. 


Hughes  Evaluation 

Problems  Encountered.  Problems  were  encountered  from  the  start  of  the 
test  which  made  a  bad  impression  on  personnel  in  the  Radar  Shop  and  made  it 
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Impossible  to  adequately  evaluate  the  system  using  the  original  test  plan.  The 
following  paragraphs  outline  the  problems  encountered. 

1.  System  Installation.  The  CMAS  hardware  was  installed  at  Offutt  AFB, 
Nebraska,  on  5  November  1*984.  Installation  was  uneventful.  However,  the 
system  required  much  more  space  in  the  Radar  Shop  than  anticipated.  The 
hardware  consisted  of  a  cabinet-mounted  central  processor  unit,  two  pedestal- 
mounted  disk  drive  units,  a  cabinet-mounted  tape  drive  unit,  and  the 
Rastertech  unit  (located  on  a  table).  These  units  required  approximately  50 
square  feet  of  floor  space,  which  necessitated  a  rearrangement  of  a  major 
section  of  the  shop.  In  addition,  the  monitor  was  installed  on  a  shelf  above 
the  workbench.  The  keyboard  and  mouse  were  placed  on  the  workbench  itself. 
The  addition  of  these  items  to  the  workbench  limited  work  space. 

2.  Heat  Generation.  The  computer  hardware  required  for  the  CMAS 
prototype"  generated  an  excessive  amount  of  heat.  The  heat  made  the  shop 
uncomfortably  warm.  In  addition,  it.  may  have  created  equipment  reliability 
problems. 

3.  System  Unreliability.  The  system  proved  to  be  very  unreliable.  This 
appeared  to  be  the  result  of  three  problems:  (a)  the  effect  of  the  heat  on 
the  hardware,  (b)  software  problems,  and  (c)  errors  in  the  technical  data. 
The  unreliability  of  the  system  made  it  very  difficult  to  use  the  system  since 
it  frequently  "froze"  making  it  necessary  to  reinitialize  the  system  and  start 
over.  This  was  very  frustrating  to  the  technicians  and  a  major  contributor  to 
their  reluctance  to  use  the  system. 

4.  Response  Times.  The  response  times  for  the  system  were  very  slow.  An 
average  of  11  seconds  was  required  to  retrieve  and  disp’ay  a  frame  of  data 
with  procedural  steps  and  locator  diagrams.  Large,  complex  illustrations 
(such  as  schematics)  required  up  to  2  minutes. 

5.  Technical  Data  Errors.  The  technical  data  contained  an  excessive 
number  of  errors.  The  errors  included  both  technical  errors  and  sequencing/ 
access  errors  (e.g.,  branching  to  the  wrong  frame).  These  errors  caused  the 
technicians  to  lose  confidence  in  the  data  and  be  reluctant  to  use  the  system. 

Hughes  Questionnaire  Results.  As  part  of  their  evaluation,  Hughes 
personnel  administered  pre-  and  post-test  questionnaires  to  the  technicians  in 
the  Radar  Shop.  The  questionnaires  were  of  the  Likert  type,  requiring  the 
respondents  to  indicate  their  degree  of  agreement  with  a  statement  on  a 
7-point  scale  (strongly  agree  to  strongly  disagree).  Analysis  of  the 
questionnaire  responses  yielded  the  following  observations: 

1.  Technicians  reported  that  the  system  increased  job  completion  time. 

2.  Technicians  agreed  with  the  statement  "I  felt  restricted  by  the 
computer."  This  feeling  was  thought  to  be  due  to  insufficient  access  options 
to  allow  movement  within  the  data  base.  The  slow  response  time  was  also 
suggested  as  a  possible  contributor  to  this  problem. 

3.  The  system's  response  time  was  too  slow  to  meet  the  user's  needs. 
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4.  The  technicians  did  not  have  trouble  reading  the  text  on  the  screen. 
The  legibility  of  the  text  and  callouts  were  rated  positively  by  the 
technicians.  The  character  size  used  for  the  system  was  judged  to  be 
appropriate  for  the  application. 

5.  The  technicians  rated  the  zoom  feature  as  helpful. 

6.  The  technicians  rated  the  schematic  detail  (readability  after  zooming) 
as  very  helpful.  It  was  recommended  that  schematics  be  presented  so  that  they 
are  initially  legible  (as  in  a  pyramid  format)  or  easily  zoomed  for  legibility. 

Hughes  Interview  Results.  At  the  end  of  the  field  test  period,  six 
technicians  who  ha'-  1  the  system  were  interviewed  by  Hughes  personnel.  The 
interviews  yie'  *nts  in  the  following  areas:  computer  software, 

computer  hardware,  ana  leninical  data  organization. 

The  following  observations  were  made  concerning  computer  software: 

1.  Technicians  found  the  CMAS  sign-on  procedures  easy  to  use. 

i 

;  2.  Technicians  found  the  menu  method  provided  to  locate  data  in  the 

system  to  be  acceptable.  Five  rated  it  as  "good,"  and  one  rated  it  as  "fair." 

3.  The  "x-searchH  feature,  which  permitted  the  user  to  retrieve  parts 
Information  by  typing  in  the  first  8  characters  of  a  part  number  or  reference 
designator,  received  a  mixed  reaction.  It  was  rated  from  "fair"  to  "very 
good." 

4.  The  "graphic  menu"  provided  for  location  of  parts  Information  was 
rated  "good"  by  four  of  the  five  who  used  it.  Comments  Included  "too  many 

\  parts,"  "some  errors,"  and  "want  overview  diagram."  The  general  response  was 

Interpreted  to  be  favorable  for  the  concept,  but  Improvements  are  required  for 
implementation. 

5.  Technicians  expressed  a  need  for  a  feature  which  would  return  the  user 
to  the  point  last  displayed  in  the  procedure  after  additional  Information  had 
been  accessed  or  displayed. 

The  following  observations  were  made  relating  to  computer  equipment: 

1.  All  six  technicians  indicated  that  they  would  prefer  dedicated 
function  keys,  labeled  with  their  functions  and  always  available  to  the  user. 

2.  The  technicians  all  agreed  that  the  keyboard  was  too  large  (taking  up 
too  much  space  on  the  workbench).  Also,  they  did  not  like  the  mouse  as 
implemented.  Five  recommended  the  use  of  a  joystick  to  control  the  cursor. 
Hughes  recommended  a  full-size  keyboard  for  system  and  data  base  maintenance 
and  a  smaller  special  purpose  keyboard  with  function  keys  and  a  joystick  or 
mouse  with  function  keys  for  use  on  the  workbench. 

3.  Technicians'  comments  suggested  a  need  for  some  method  of  reducing  the 
glare  on  the  screen  and  improved  placement  of  the  display. 
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The  following  observations  were  made  relating  to  technical  data 
organization: 

1.  Although  the  response  speed  of  the  system  was  seen  as  the  major 
drawback  in  user  acceptance,  user  acceptance  of  the  use  of  color,  system 
resolution,  letter  font,  spacing,  and  viewing  distance  was  good. 

2.  Technicians  had  mixed  reactions  to  the  three-track  concept  (as  applied 
in  the  CMAS  prototype).  Two  stated  that  Track  3  (most  detailed)  was 
"useless."  Another  could  see  no  need  for  Tracks  2  and  3. 

3.  All  technicians  preferred  to  have  only  those  callouts  in  the  graphic 
that  are  referenced  in  the  procedure.  As  implemented,  the  graphics  contained 
all  callouts  whether  or  not  the  callouts  were  referenced  in  the  frame  (see 
Figure  35). 


All  Callouts  Shown. 

As  a  result  of  the  problems  discussed  above,  the  CMAS  prototype  system 
failed  to  achieve  acceptance  by  the  technicians.  This  resulted  in  the 
technicians'  being  unwilling  to  use  the  system  for  their  day-to-day 
maintenance  of  the  testbed  system,  as  required  to  implement  the  original  test 
plan.  It  was  decided  that  an  alternate  approach  would  be  required  to 
adequately  evaluate  the  system  and  obtain  the  maximum  benefit  from  the  study. 
A  substitute  evaluation  plan  was  then  developed  by  AFHRL,  and  the  Hughes 
evaluation  was  limited  to  posttest  interviews  and  questionnaires  which  were 
collected  following  the  AFHRL  evaluation. 


AFHRL  Evaluation 

Prior  to  implementation  of  the  AFHRL  evaluation,  AFHRL  personnel  conducted 
a  thorough  review  of  the  data  presented  on  the  system  in  an  effort  to 
eliminate  as  many  errors  as  possible  from  the  data.  Technical  errors  were 
corrected  by  Hughes  personnel. 


Procedures.  With  the  assistance  of  an  experienced  technician,  four 
problems  were  identified  which  could  be  used  for  the  test.  Problem  selection 
was  based  upon  the  following  criteria:  (a)  the  task  could  not  require  more 
than  1  hour  to  complete;  (b)  accurate  data  for  completing  the  task  must  be 
available  in  the  CMAS  data  base;  (c)  the  task  could  be  accomplished  without 
damage  to  the  equipment  (since  there  was  not  time  to  obtain  dedicated 
equipment  or  spares  for  the  test);  and  (d)  tasks  for  each  category  (checkout 
and  remove/replace)  must  be  comparable  in  difficulty  and  time  to  complete. 
Two  checkout  and  two  remove/replace  tasks  were  selected.  The  checkout  tasks 
involved  performing  the  checkout  procedure  until  two  sensitivity  out-of¬ 
tolerance  conditions  were  identified.  The  remove  and  replace  tasks  involved 
removing  two  circuit  boards  from  the  intermediate  frequency  (IF)  module.  The 
tasks  were  judged  to  be  adequate  to  give  a  general  indication  of  the  ability 
of  the  CMAS  to  support  the  performance  of  maintenance  on  the  testbed  system. 
However,  it  did  not  provide  a  thorough  test  of  the  system  and  its  ability  to 
support  maintenance.  The  problems  did  provide  technicians  with  sufficient 
experience  on  the  system  to  permit  them  to  provide  their  personal  evaluation 
of  the  system  and  recommendations  for  improvements. 

Each  task  was  performed  by  12  technicians.  Six  of  the  technicians  were 
classified  by  a  supervisor  as  having  high  experience  on  the  system,  and  six 
were  classified  as  having  low  experience  on  the  system.  Each  technician 
performed  one  checkout  and  one  remove/replace  task  with  the  CMAS,  and  one 
checkout  and  one  remove/replace  task  with  the  paper-based  technical  order. 
The  tasks  were  performed  using  the  Shop's  test  bench  for  the  AN/APX-64. 
Testing  was  done  on  a  noninterference  basis.  Testing  conditions  were 
"semi-controlled"  in  that  the  test  subject  was  isolated  so  interference  was 
limited  as  much  as  possible.  However,  some  interruptions  did  occur  due  to  the 
shop  environment  and  duties  of  the  technicians.  The  performance  of  each 
technician  was  observed  by  one  test  administrator  who  noted  problems, 
evaluated  performance,  and  recorded  performance  times.  Performance  times  were 
adjusted  to  account  for  delays  from  interruptions  or  system  problems.  Each 
technician  was  allowed  to  continue  work  on  the  task  until  completion  or  until 
he  gave  up. 

Questionnaires  were  administered  to  the  technicians  following  the 
completion  of  the  performance  tests.  The  questions  were  open-ended. 

Performance  Test  Results.  Data  wc^e  collected  on  successful  completion 
and  time  to  complete  each  task.  All  objects  were  able  to  successfully 
complete  each  task.  The  times  o  perform  each  task  are  shown  in  Table  4. 
Mean  performance  times  for  each  task  are  shown  by  group  (high  versus  low 
experience).  The  times  to  perform  the  tasks  using  the  CMAS  were  longer  in 
most  cases  than  for  paper  TOs.  There  is  very  little  difference  in  the  mean 
times  shown  for  the  checkout  task  for  the  high-experience  group.  Examination 
of  the  individual  times  indicates  that  this  is  due  to  the  excessive  time  for 
one  individual  when  using  the  TO.  This  can  be  explained  by  the  fact  that  the 
individual  was  a  supervisor  with  several  years  of  experience  on  the  system. 
He  had  not  actually  worked  on  the  system  for  some  time.  He  performed  the  task 
first  with  the  TO.  This  served  as  a  refresher  for  him  and  probably  aided  his 
performance  with  the  CMAS.  If  the  performance  time  for  this  individual  for 
that  task  (checkout  using  the  TO)  is  eliminated,  the  mean  times  for  the  high- 
experience  group  are  29.6  minutes  with  the  CMAS  and  15.8  minutes  with  the  TO. 


Table  4.  Task  Performance  Times  (In  Minutes) 


Subject 

CO/CMAS 

CO/TO 

RR/CMAS 

RR/TO 

H-l  ' 

■■KtH'IiHI 

M  M  il 

H-2 

13.00 

14.00 

9.00 

4.00 

H-3 

32.00 

13.00 

4.00 

8.00 

H-4 

35.00 

11.00 

7.00 

7.00 

H-5 

37.00 

95.00 

8.00 

4.00 

H-6 

23.00 

33.00 

14.00 

5.00 

Total 

185.00 

174.00 

47.00 

33.00 

Mean 

30.83 

29.00 

7.83 

5.50 

L-l 

58.00 

24.00 

14.00 

6.00 

L-2 

38.00 

14.00 

12.00 

6.00 

L-3 

32.00 

25.00 

7.00 

7.00 

L-4 

30.00 

34.00 

10.00 

11.00 

L-5 

58.00 

-15.00 

14.00 

4.00 

L-6 

52.00 

12.00 

6.00 

7.00 

Total 

268.00 

124.00 

63.00 

41.00 

Mean 

44.67 

20.67 

10.50 

6.83 

Grand 

total 

453.00 

298.00 

110.00 

74.00 

Grand 

mean 

37.75 

24.83 

9.17 

6.17 

H  *  high  experience ;  L  * 

low  experience. 

The  data  presented  In  Table  4  are  provided  to  give  a  general  idea  of  the 
Impact  of  CMAS  on  performance.  However,  caution  should  be  used  in  generaliz¬ 
ing  from  these  data  due  to  the  lack  of  rigor  In  the  test  procedures.  A  formal 
inferential  statistical  analysis  was  not  made  of  the  data  since  there  were 
sufficient  Irregularities  (Interferences,  etc.)  in  the  data  collection  proce¬ 
dures  that  assumptions  required  by  formal  analysis  procedures  could  not  be  met. 

Questionnaire  Results.  Analysis  of  the  technicians'  responses  to  the 
questionnaire  yielded  the  following  observations: 

1.  Of  the  11  technicians  who  completed  the  questionnaire,  10  stated  that 
the  system  response  time  was  too  slow.  One  Indicated  that  he  believed  using 
the  system  could  be  faster  than  using  the  TO  If  the  system  response  time  were 
faster. 

2.  The  technicians  found  that  the  readability  of  the  display  was  satisfac¬ 
tory.  The  font  sizes  used  to  present  textual  materials  and  to  identify 
callouts  were  considered  acceptable. 

3.  Six  of  the  technicians  found  the  mouse  easy  to  use.  Three  said  It  was 
hard  to  learn  to  use  or  was  confusing.  One  simply  did  not  like  the  mouse. 

4.  The  zoom  feature  was  found  to  be  useful  by  most  technicians.  Only  one 
technician  reported  any  difficulty  In  using  the  zoom  feature. 
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5.  The  scroll  feature  received  mixed  reviews.  Only  one  technician  rated 
it  as  "good."  Other  comments  included:  "it  is  impractical"  or  "not  needed"; 
"it  is  too  slow";  "couldn't  figure  it  out";  and  "scrolls  backward." 

6.  Eight  of  the  technicians  indicated  that  the  highlighting  feature  was 
helpful.  One  found  it  hard  to  use  and  two  did  not  use  it. 

7.  The  techniques  for  accessing  the  data  received  mixed  reviews.  Some 
felt  that  they  were  convenient  and  easy  to  use.  Others  felt  that  it  was  too 
difficult  to  get  from  one  place  to  another  in  the  data  base. 

8.  Reactions  to  the  use  of  color  on  the  system  were  mixed.  Most  found  it 
to  be  "acceptable"  or  "good."  Some  indicated  that  it  is  not  necessary  for 
this  type  of  system. 

9.  The  resolution  of  the  display  was  rated  "good"  or  "adequate"  by  all  of 
the  technicians  who  responded. 

10.  Glare  on  the  display  was  seen  as  a  problem  by  most  of  the 
technicians.  However,  five  of  the  technicians  indicated  the  use  of  a  shade 
adequately  alleviated  the  problem. 

11.  The  arrangement  of  the  display,  keyboard,  and  mouse  on  the  workbench 
was  seen  as  a  problem  by  many.  The  location  of  the  display  on  a  shelf  above 
the  workbench  was  considered  satisfactory  by  most  technicians.  However, 
several  suggested  repositioning  it  to  reduce  glare  and  to  improve  ease  of 
viewing.  The  keyboard  and  mouse  were  seen  as  taking  up  too  much  space  on  the 
workbench,  making  the  workbench  too  cluttered.  Several  suggestions  were 
offered.  Including  the  use  of  a  smaller  keyboard  with  only  required  functions 
provided,  the  use  of  voice  control,  and  the  use  of  a  joystick  in  place  of  the 
mouse. 

12.  The  level  of  detail  of  the  graphics  was  rated  "adequate"  by  eight  of 
the  nine  technicians  who  responded  to  the  question.  One  indicated  that  the 
graphics  needed  depth. 

13.  The  use  of  three  levels  of  detail  (tracks)  received  mixed  reactions. 
Five  of  the  nine  subjects  who  answered  the  question  Indicated  that  the 
multiple  levels  of  detail  were  helpful.  The  remaining  four  did  not  believe 
so.  Six  of  the  eight  subjects  who  Indicated  a  preferred  track  selected  Track 
2.  Two  stated  that  they  preferred  Track  2.  None  of  the  technicians  preferred 
Track  3;  however,  two  indicated  It  would  be  useful  as  a  training  aid  or  for 
someone  with  no  experience. 

14.  When  asked  if  using  the  system  caused  fatigue,  six  technicians 
indicated  that  it  either  caused  no  fatigue  or  caused  no  more  fatigue  than  the 
paper  TO.  Three  Indicated  that  they  experienced  some  eyestrain.  One  noted 
that  "it  probably  could  get  to  me  after  awhile." 


Other  Observations 


Throughout  the  field  test  period,  AFHRL  personnel  on-site  continually 
observed  the  use  of  the  system  to  identify  problems,  potential  solutions,  and 


85 


possible  improvements  of  the  system.  Their  observations  were  provided  to 
Hughes  personnel  to  solicit  their  comments  and,  when  possible,  to  modify  the 
system.  The  principal  comments/problems/recommendations  provided  to  Hughes 
personnel  are  provided  below. 

1.  Graphic  zooming  is  limited  to  the  size  and  shape  of  the  window.  Most 
graphics  are  oblong  rectangles.  Thus,  when  an  object  is  zoomed,  it  is  forced 
into  the  (same)  oblong  window  (regardless  of  the  shape  of  the  object  to  be 
zoomed).  The  result  is  that  the  drawing  becomes  distorted.  This  is  true  for 
both  schematics  and  equipment.  (For  example,  this  situation  occurs  when  a 
square  area  of  the  schematic  or  a  square  object  in  an  equipment  drawing  is 
zoomed  and  forced  to  fit  into  a  rectangular  window.) 

2.  If  the  graphics  mode  is  used  (which  allows  scrolling  and  zooming),  the 
system  returns  to  the  start  of  the  module  (start  of  the  procedure)  when 
exiting,  regardless  of  where  the  graphics  mode  was  entered.  As  a  result,  the 
user  may  be  forced  to  page  through  several  frames  to  get  back  to  where  he 
started  (where  the  graphics  mode  was  entered). 

3.  In  the  zoom  mode,  the  system  will  zoom  each  time  button  #1  on  the 
mouse  is  pressed.  Button  #2  must  be  pressed  to  get  out  of  the  zoom  mode.  If 
the  user  forgets  and  tries  to  recall  the  menu  (by  pressing  button  #1)  without 
pressing  button  #2,  the  system  will  attempt  to  zoom  an  infinitely  small  area. 
The  system  will  freeze,  forcing  the  user  to  reset  the  system  and  start  over. 

4.  In  the  graphics  mode,  if  the  "display  full  screen"  option  is  selected, 
the  graphic  (which  was  designed  for  an  oblong  rectangular  window)  is 
distorted.  (The  full-screen  display  window  is  square.) 

5.  When  a  graphic  is  presented,  an  option  is  given  to  "manipulate  the 
graphic."  The  user  is  instructed  to  enter  "Y"  to  manipulate  the  graphic.  The 
system  does  not  say  what  to  do  if  the  graphics  mode  is  not  wanted.  Any  other 
key  will  clear  the  graphic  mode  (although  the  user  is  not  told  this).  This 
means  that  any  other  key  must  be  pressed,  and  then  NEXT  must  be  pressed  to  go 
to  the  next  frame.  A  problem  occurs  when  the  user  is  in  a  branching  frame  and 
must  respond  "yes"  or  "no."  If  the  user  forgets  to  clear  the  graphic  question 
and  answer  "yes,"  he  is  forced  into  the  graphics  mode.  Then,  when  the 
graphics  mode  is  exited,  the  user  is  returned  to  the  start  of  the  module  and 
must  page  through  several  frames  to  get  back  to  where  he  began. 

6.  There  is  no  way  of  directly  accessing  a  schematic  without  paging 
through  the  Theory  of  Operation  section  of  the  TO. 

7.  In  many  cases  (such  as  Theory  of  Operation),  an  individual  graphic 
(such  as  a  block  diagram)  supports  several  frames  of  text.  However,  the 
graphic  cannot  be  manipulated  until  the  user  has  reached  the  last  frame  of 
text.  Thus,  if  the  user  wants  to  manipulate  the  graphic  to  examine  the 
drawing  to  answer  a  question  brought  about  by  the  first  frame,  he  must  page  to 
the  last  frame.  Then  the  text  is  not  available  for  reference.  This  could  be 
a  problem  since  many  of  the  drawings  (block  diagrams,  schematics,  etc.)  are 
not  readable  as  initially  presented. 

8.  Several  frames  are  required  to  present  some  blocks  of  text.  If  an 
attempt  is  made  to  backspace  before  reaching  the  end  of  the  block  (end  of 
module),  the  system  will  freeze,  forcing  the  user  to  reboot  and  start  over. 


86 


9.  The  formatting  of  procedural  frames  is  inconsistent.  Some  present  a 
full  page  of  steps;  some  only  one  or  two  steps  per  frame.  Some  use  tables  to 
present  cable  setup  instructions;  others  present  the  setup  procedures  as  a  set 
of  step-by-step  instructions. 

10.  The  use  of  color-coding  in  equipment  drawings  makes  it  difficult  to 
distinguish  some  components  (due  to  poor  contrast). 

11.  The  equipment  drawings  contain  callouts  for  every  component, 
regardless  of  whether  the  item  is  called  out  in  the  frame  of  the  procedure. 
Thus,  the  technician  must  search  through  all  of  the  callouts  to  find  the  ones 
of  interest.  Some  of  the  drawings  are  quite  cluttered  with  callouts. 

12.  Many  of  the  equipment  drawing  callouts  are  too  small  to  be  read 
easily.  In  one  case,  the  callouts  appear  as  "dots"  and  are  completely 
illegible.  If  the  zoom  is  used  to  read  the  callout,  the  drawing  is 
distorted.  Then,  when  the  graphic  mode  is  exited,  the  user  is  returned  to  an 
earlier  part  of  the  procedure  and  must  page  through  several  frames  to  return 
to  where  he  started. 

13.  With  the  exception  of  checkout  procedures,  troubleshooting 
procedures,  and  some  IPB  information,  the  technical  data  on  the  system  are 
direct  copies  from  the  Technical  Order.  Thus,  with  these  exceptions,  the 
system  is  simply  a  "page-turner,"  not  much  different  than  AFHRL  has  strongly 
opposed  in  the  past.  One  of  the  prime  goals  of  the  program  has  always  been  a 
system  that  is  more  than  just  a  page-turner. 

14.  Drawings,  schematics,  block  diagrams,  etc.  are  not  identified.  This 
is  not  critical  for  procedural  data,  but  it  is  for  other  types  of  data.  For 
example.  Theory  of  Operation  contains  text  with  a  block  diagram  at  the  bottom 
of  the  frame.  The  diagram  is  not  identified.  Therefore,  the  user  cannot  be 
sure  what  it  is. 

15.  Many  illustrations  supporting  procedural  text  do  not  have  the  option 
of  manipulating  graphics. 

16.  Many  frames  have  too  much  text  (i.e.,  the  frame  is  full  of  unbroken 
text).  This  makes  it  difficult  to  read  and  increases  eyestrain. 

17.  There  is  no  direct  access  to  "pool  data"  as  originally  required.  If 
a  technician  using  a  procedure  wants  to  view  a  schematic,  he  must  exit  from 
the  procedure,  go  back  to  the  table  of  contents,  find  the  appropriate  section, 
and  then  page  through  the  section  until  he  finds  the  schematic  (which  is  not 
labeled). 

18.  The  MIDAS  coding  system  is  not  used  as  required. 

19.  The  frames  are  not  labeled.  The  user  cannot  tell  where  he  is  in  the 
system.  This  could  be  a  problem  if  the  user  is  interrupted  and  goes  back 
later,  or  if  a  new  technician  takes  over  part  way  through  the  task.  If  he  is 
in  a  section  that  has  multiple  tracks,  he  cannot  be  sure  which  track  he  is  In. 

20.  The  mouse  is  very  awkward  to  use. 
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21.  To  switch  tracks  within  a  procedure,  the  user  must  press  "d"  for  more 
detail  and  "a"  for  less  detail.  The  use  of  "a"  is  not  a  logical  choice. 

22.  Some  of  the  IPB  information  is  keyed  to  the  paper  TO.  For  example, 
the  parts  list  refers  to  Figure  X  in  the  paper  IPB.  It  would  be  necessary  to 
go  to  the  paper  IPB  to  use  it.  This  is  not  acceptable  since,  in  a  fielded 
system,  there  would  be  no  paper  IPB. 

23.  The  typical  delay  between  the  time  the  NEXT  key  is  pushed  and  the 
presentation  of  the  next  procedural  frame  is  10  to  12  seconds.  More  complex 
graphics  take  somewhat  longer. 

24.  It  would  be  difficult  to  locate  information  in  the  system  if  the  user 
does  not  know  what  section  of  the  TO  it  is  in.  The  only  indexing  techniques 
used  are  those  available  in  the  TO.  If  a  technician  would  have  trouble 
finding  the  information  in  the  TO,  he  would  have  trouble  finding  it  in  the 
CMAS.  In  fact,  it  probably  would  be  harder  to  find  information  in  the  CMAS 
than  in  the  paper  TO.  It  is  easier  and  quicker  to  page  through  a  paper  TO 
than  to  page  through  the  CMAS  (which  takes  10  to  12  seconds  per  frame). 

25.  When  a  fault  is  identified,  the  troubleshooting  procedure  does  not 
give  an  option  to  go  directly  to  corrective  maintenance  instructions.  It  is 
necessary  to  back  out  of  the  system  and  go  to  the  index  to  find  the  correct 
instructions.  Similarly,  the  system  does  not  provide  direct  access  to  parts 
information  or  other  pool  type  information  from  a  procedural  frame. 

26.  Questions  to  be  answered  by  the  technicians  are  formatted  poorly. 

27.  The  CMAS  Main  Menu  is  not  user-friendly.  If  the  technician  wants 
data  from  Section  1,  the  logical  key  selection  would  be  "1";  but  on  the  CMAS 
system,  the  user  must  press  "a." 

28.  Substeps  are  numbered  just  like  the  callouts.  For  example,  "(1)  turn 
oscillator  switch  (32)  to  on."  The  (1)  should  have  been  replaced  with  a.,  b., 
c.,  etc. 

29.  Proper  headings  are  not  repeated  for  frames  continued  on  a  second 
frame. 

30.  There  are  several  places  where  the  BACK  key  does  work.  In  other 
cases,  pressing  the  BACK  key  will  freeze  the  system. 

Seven  of  the  problems  described  above  were  corrected  (although  not  always 
optimally)  early  in  the  field  test  period  and  before  the  start  of  the  AFHRL 
evaluation.  These  were  items  3,  6,  8,  12,  15,  26,  and  30.  There  were 
insufficient  time  and  resources  to  resolve  the  remaining  problems.  Many  of 
these  problems  would  have  required  extensive  revision  of  the  CMAS  hardware  and 
software  or  extensive  rewrites  of  the  technical  data. 


Preliminary  Data  Base  Design  Effort 

During  the  Initial  phases  of  the  CMAS  project,  emphasis  was  placed  upon 
developing  effective  techniques  for  delivering  and  presenting  technical  data. 
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However,  the  investigators  were  aware  that  the  problems  of  creating  and 
maintaining  the  data  base  required  to  support  a  CMAS  would  be  difficult  to 
resolve.  The  magnitude  of  the  problems  became  apparent  during  the  CMAS  I 
effort  when  the  complexities  of  creating  a  technical  data  base  which  is 
technically  accurate  were  encountered.  Two  problem  areas  were  identified: 

1.  Data  Creation.  Creating  technical  data  for  the  CMAS  I  required 
creating  not  only  the  technical  data  itself  but  the  complex  codes  required  to 
control  the  computer's  retrieval  and  presentation  of  the  data.  Technical  data 
authors  were  required  to  input  the  complex  codes  by  hand  with  only  limited 
help  from  technical  data  authoring  software  developed  under  the  NTIPS 
program.  The  result  was  that  the  creation  of  the  data  was  expensive,  and  the 
data  contained  many  system  control  code  errors  which  caused  many  of  the 
technical  data  presentation  problems  encountered  in  the  CMAS  I  evaluation. 

2.  Technical  Data  Presentation.  The  technical  data  and  technical  data 
presentation  software  developed  in  the  CMAS  system  could  be  operated  on  only 
one  computer  system.  When  it  was  realized  that  the  computer  system,  as 
configured,  was  inadequate  to  support  the  CMAS,  it  was  not  possible  to  change 
to  a  more  suitable  system  because  the  software  and  data  base  could  not  be  used 
with  any  other  system.  This  weakness  prevented  transferring  to  another  system 
which  would  have  improved  the  chances  of  the  CMAS  I  being  successful.  It  also 
highlighted  another  consideration:  the  necessity  of  designing  the  data  base 
so  that  it  would  not  be  limited  to  presentation  on  one  system.  This  is 
essential  so  that  when  Improved  computer  systems  become  available,  the  data 
can  be  presented  on  the  new  equipment  or  used  in  other  ways  without  an 
extensive  rewrite. 

These  observations,  along  with  similar  concerns  expressed  by  other  AFHRL 
personnel  working  in  support  of  the  ATOS  program,  led  to  a  decision  to 
establish  an  effort  to  study  the  data  base  design  issue.  The  Rockwell 
contract  was  modified  to  add  two  additional  requirements:  (a)  to  conduct  an 
analysis  of  the  data  base  requirements,  and  (b)  to  develop  a  coding  scheme  for 
preparing  the  data  which  produces  a  "neutral"  data  base  which  could  be 
presented  on  any  computer  with  the  required  capabilities. 

The  Rockwell  effort  included  a  detailed  analysis  of  Air  Force 
specifications  for  intermediate  level  technical  data.  The  analysis  identified 
all  types  of  technical  data  which  must  be  supported  by  an  ATDPS.  The 
requirements  for  each  type  of  data  were  then  analyzed  to  Identify  each  element 
of  data  and  the  attributes  of  each  element.  The  attributes  were  then 
categorized  to  provide  a  listing  of  elements  with  common  attributes.  The 
categories  provided  the  necessary  source  information  for  the  development  of  a 
preliminary  coding  scheme. 

The  coding  scheme  was  developed  under  subcontract  by  Datalogics,  Inc.  It 
was  based  upon  the  Standard  Generalized  Markup  Language  (SGML)  being  used  with 
the  ATOS  program.  SGML  was  selected  as  a  starting  point  to  ensure 
compatibility  with  ATOS.  The  SGML  coding  scheme  tags  each  element  of  data 
(e.g.,  a  paragraph,  part  number,  or  title)  with  an  Identifier  specifying  the 
type  of  information  and  relationships  to  other  data  elements  (e.g.,  a 
paragraph  may  have  codes  specifying  the  chapter  It  belongs  to,  the  preceding 
paragraph,  or  an  illustration  which  must  accompany  the  paragraph). 


Once  the  technical  information,  with  its  corresponding  codes,  is  developed 
and  stored,  it  can  be  formatted  for  printing  or  for  display  on  a  computer. 
This  can  be  accomplished  using  special  purpose  software  designed  to  format  the 
data  according  to  the  characteristics  of  the  specified  display  device  and 
predefined  formatting  rules.  For  example,  if  the  data  element  is  identified 
as  a  header  element,  the  data  will  be  presented  (or  printed)  in  the  location 
on  the  display  (or  page)  specified  by  the  formatting  rules  and  display  (or 
page)  characteristics.  This  approach  will  make  it  possible  to  display 
technical  data  on  a  variety  of  systems  without  modification  to  the  data  base. 
It  will  make  it  possible  to  print  the  data,  if  desired,  or  use  it  in  other 
ways  without  modifying  the  data  base. 

The  results  of  the  analysis  and  coding  specification  have  provided  the 
starting  point  for  an  in-house  effort.  The  objective  of  this  effort  is  to 
extend  and  refine  the  set  of  identified  data  elements  and  to  develop  software 
for  a  system  to  author  and  present  technical  data  using  a  neutral  data 
representation  on  an  ATDPS.  This  effort  is  described  briefly  in  Section  VI. 
A  more  detailed  description  is  presented  in  Link  et  al.  (1987). 


Specification  Development 

The  contract  Statement  of  Work  required  Hughes  to  develop  two  draft 
specifications  for  use  in  procuring  a  CMAS  system.  The  first  specification 
was  to  establish  the  requirements  for  technical  data  to  be  presented  on  the 
system.  The  second  was  to  provide  the  functional  requirements  for  system 
hardware  and  software.  The  specifications  were  to  be  based  upon  the  CMAS 
prototype  and  lessons  learned  in  the  field  test. 

After  the  field  test  was  completed,  Hughes  developed  two  draft 
specifications  (Hughes  Aircraft  Company,  1985c  and  1985d).  However,  the 
specifications  were  considered  inadequate  in  many  areas  and  were  not 
published.  The  primary  problem  with  the  technical  data  specification  was  that 
it  provided  only  very  general  requirements  for  the  technical  data  and  did  not 
adequately  specify  requirements  for  the  content  of  the  technical  data.  The 
main  problem  with  the  functional  specification  was  that  it  was  primarily  a 
system  hardware  specification,  rather  than  a  functional  specification.  The 
specification  of  hardware  requirements  was  beyond  the  scope  of  the  effort  and 
dependent  upon  many  operational  considerations  that  could  not  be  accurately 
determined  at  the  time. 


Discussion  and  Conclusions 

Although  the  prototype  CMAS  developed  under  this  effort  fell  far  short  of 
the  program  goals,  it  did  provide  much  valuable  information  and  many  "lessons 
learned."  The  effort  did  establish  the  feasibility  of  an  automated  system  for 
presenting  technical  data.  Technicians  were  able  to  perform  representative 
maintenance  tasks  without  difficulty  using  data  presented  on  the  CMAS.  Even 
though  the  system  did  not  receive  general  acceptance  by  the  technicians,  there 
were  clear  indications  that  they  would  readily  accept  an  automated  system  once 
certain  design  flaws  (such  as  response  time)  were  corrected. 
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The  effort  provided  important  lessons  for  the  further  development  and 
refinement  of  future  automated  technical  data  systems.  It  clearly  reinforced 
AFHRL's  long-held  position  that  user  acceptance  is  paramount  for  automated 
systems  such  as  CMAS.  The  CMAS  did  not  gain  the  acceptance  of  the  technicians 
primarily  due  to  its  slow  response  time  and  the  presence  of  features  which 
made  it  difficult  to  use  (not  user-friendly).  The  criticality  of  user 
acceptance  makes  it  essential  that  potential  users  be  involved  in  the  design 
of  future  systems.  Technicians  participating  in  the  design  studies  had  many 
valuable  inputs  which,  if  they  had  been  implemented,  would  have  greatly 
improved  the  system  and  acceptance  by  the  technicians. 

The  results  of  the  CMAS  field  test  provided  the  basis  for  a  number  of 
design  guidelines  and  recommendations  for  developing  future  systems.  The 
results  clearly  supported  the  following  recommendations  for  the  development  of 
future  automated  technical  data  systems: 

1.  The  system  must  be  user-friendly.  To  ensure  user-friendliness,  the 
system  should  provide: 

a.  A  rapid  response  time.  The  time  to  retrieve  a  frame  of 
procedural  data  should  not  exceed  5  seconds  and  preferably,  should  be  much 
less. 


b.  An  easy  means  for  locating  and  accessing  data. 

c.  A  simple  means  of  moving  about  in  the  data  base  (move  from  one 
section  to  another,  retrieve  support  data,  retrieve  parts  information,  etc.). 

d.  An  indicator  showing  the  user's  location  in  the  data  base. 

2.  Multiple  levels  of  detail  (tracks)  should  be  provided  for  procedural 
data.  The  findings  of  the  field  test  suggested  that  two  tracks  are  adequate 
for  application  with  technical  data  for  electronic  systems.  However,  it  is 
unknown  whether  two  tracks  are  adequate  to  support  the  maintenance  of 
mechanical  systems.  Further  research  is  needed  to  examine  this  issue. 

3.  The  "job  guide"  format  used  in  this  study  proved  to  be  an  effective 
format  for  presenting  procedural  data  via  an  automated  technical  data  system. 
When  this  format  is  used,  only  those  callouts  referenced  in  the  text  of  the 
frame  should  be  shown  on  the  supporting  illustration. 

4.  The  use  of  a  pyramiding  approach  to  the  presentation  of  complex 
diagrams,  such  as  schematics,  is  recommended.  The  diagrams  should  be 
presented  such  that  each  drawing  Is  of  a  size  which  is  readable  as  first 
displayed. 

5.  The  use  of  color-coding  to  identify  functional  segments  of  schematics 
or  similar  diagrams  is  recommended.  The  value  of  other  uses  of  color-coding 
(e.g.,  to  Identify  classes  of  components  such  as  switches)  on  locator  diagrams 
is  uncertain  and  requires  further  research. 

6.  A  resolution  level  of  approximately  50  pixels  per  inch  was  sufficient 
for  graphics  required  to  support  electronic  systems.  However,  it  Is  unknown 
whether  this  level  of  resolution  is  sufficient  for  graphics  supporting 
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maintenance  of  mechanical  systems.  Graphics  for  mechanical  systems  usually 
are  more  complex  and  require  the  display  of  graphic  elements  (such  as  circles, 
splines,  and  diagonal  lines)  which  are  more  difficult  to  represent  on  a 
low-resolution  computer  display.  Research  is  needed  to  clarify  this  question. 

7.  Technical  data  for  presentation  on  an  automated  technical  data  system 
must  be  100%  accurate.  Thorough  validation  and  verification  of  all  data  are 
essential.  This  is  especially  critical  for  procedural  data  intended  for  use 
by  personnel  with  limited  experience,  since  the  inexperienced  technician  does 
not  have  the  background  to  compensate  for  errors  in  the  data.  The  problem  of 
ensuring  complete  accuracy  of  the  technical  data  is  much  more  complex  for 
automated  technical  data  since,  not  only  does  the  information  have  to  be 
technically  accurate,  the  branching  instructions  must  lead  to  the  correct  next 
frame.  The  codes  which  tell  the  computer  which  frame  to  go  to  next,  which 
frame  to  go  to  if  the  user  wants  to  back  up,  etc.  must  be  accurate. 
Otherwise,  the  user  could  be  sent  to  a  totally  irrelevant  part  of  the  data 
base  (never  to  return)  and  be  forced  to  abort  and  start  over  (assuming  that  he 
realizes  that  an  error  has  been  made). 

In  retrospect,  one  can  identify  a  number  of  reasons  for  the  failure  of  the 
project  to  develop  a  prototype  CMAS  system  which  met  the  design  goals.  A 
major  mistake  made  by  both  the  Government  and  the  contractors  was  to 
underestimate  the  complexity  of  the  task  and  the  resources  required  to 
accomplish  it.  This  led  to  too  much  of  the  resources  and  time  being  spent  on 
tasks  of  lesser  importance  (such  as  the  identification  of  tasks  for  the  sample 
data  base),  leaving  insufficient  time  and  funds  to  build  the  actual 
prototype.  Technical  misjudgements  were  made  also.  Perhaps  the  most  serious 
was  the  selection  of  the  Rastertech  color  graphics  terminal.  The  effort 
required  to  make  the  terminal  work  with  the  MODCOMP  computer  and  the 
NTIPS/Simpler  software  used  up  an  excessive  amount  of  resources  and  time.  The 
"cludge"  system  which  resulted  was  unable  to  meet  response  time  requirements 
and  was  a  major  cause  of  its  failing  to  achieve  user  acceptance. 

The  net  result  of  these  and  similar  problems  was  that  when  it  came  time  to 
develop  the  actual  prototype  and  technical  data  for  the  field  test,  there  were 
inadequate  time  and  resources  left  to  do  it  right.  As  a  result,  many 
shortcuts  (e.g.,  including  all  callouts  on  a  locator  graphic  whether 
referenced  in  frame  or  not)  were  taken,  and  features  known  to  be  desirable 
(e.g.,  pyramid  graphics)  were  not  included.  In  addition,  there  were 
insufficient  funds  and  time  to  create  and  validate  the  technical  data  and 
debug  the  system.  The  result  of  these  and  other  factors  (e.g.,  failure  to 
apply  sound  human  factors  principles)  was  a  system  which  was  not  usable  and 
not  acceptable  to  the  technician. 

The  Hughes/Rockwell,  Behavioral  Technology,  and  Unified  Industries  efforts 
have  established  a  solid  background  for  the  development  of  an  effective  proto¬ 
type  CMAS  system.  Immediately  following  the  field  test,  an  AFHRL  in-house 
effort  was  established  to  develop  a  second  prototype  CMAS.  This  effort  is 
described  in  the  next  section. 


V.  CMAS  II  DEVELOPMENT  AND  EVALUATION 

The  CMAS  I  (Rockwell/Hughes)  effort  had  demonstrated  the  basic  concepts 
for  an  automated  technical  data  presentation  system  and  shown  that  such  a 
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system  is  feasible.  However,  it  was  realized  that  significant  improvements  in 
the  system  were  essential  before  it  would  be  acceptable  to  technicians  and 
usable  in  an  operational  environment.  An  in-house  effort  was  therefore 
established  to  develop  and  demonstrate  an  improved  CMAS  system  which  would  not 
have  the  limitations  of  the  CMAS  I  and  would  achieve  ready  acceptance  by  the 
technicians.  This  in-house  effort  developed  a  second  prototype  system  known 
as  CMAS  II. 

The  purpose  of  the  CMAS  II  effort  was  to  develop  and  demonstrate  an 
improved  CMAS  which:  (a)  did  not  have  the  limitations  of  the  CMAS  I,  (b)  would 
be  well  accepted  by  the  user  (technician),  and  (c)  incorporated  features  which 
were  practical  for  an  operational  system.  In  developing  the  CMAS  II,  emphasis 
was  placed  upon: 

1.  Improving  response  time  (not  to  exceed  5  seconds  for  presentation  of  a 
procedural  frame). 

2.  Improving  the  MMI  to  make  the  system  easier  to  use,  more  flexible,  and 
user-friendly. 

3.  Improving  data  access  techniques  to  make  it  easier  for  the  user  to 
locate  information  and  move  around  in  the  data  base. 

The  CMAS  II  prototype  was  developed  to  demonstrate  and  test  the  potential 
of  an  automated  technical  data  system.  It  was  not  intended  for  actual 
operational  implementation.  For  the  system  to  be  suitable  for  operational 
use,  additional  work  would  be  required  to  simplify  the  process  of  creating 
technical  data  for  presentation  on  the  system  and  to  improve  the  capability  of 
the  system  to  store  and  present  graphics. 

The  development  of  the  CMAS  II,  descriptions  of  the  hardware  and  software, 
development  of  the  technical  data,  and  the  results  of  a  field  demonstration 
are  discussed  in  the  following  sections. 


System  Description 


Hardware 


The  hardware  chosen  for  CMAS  II  was  the  Grid  Compass  II  computer  Model 
1139  (Figure  36),  with  a  Grid  Winchester  disk  Model  2101.  The  Grid  Compass  II 
computer  was  selected  because  it  had  the  capabilities  required  to  support  the 
CMAS  II  and  was  readily  available.  The  Grid  Compass  II  was  an  off-the-shelf, 
lap-top  microcomputer  that  was  originally  designed  as  a  general-purpose 
business  computer.  However,  its  small  size  and  capability  to  present  graphics 
made  it  an  excellent  choice  for  the  CMAS  II  prototype.  The  specifications  of 
the  Grid  Compass  II  computer  and  disk  drive  are  presented  in  Table  5. 

The  Grid  Compass  II  computer  and  Winchester  disk  were  the  only  hardware 
required  for  the  CMAS  II  prototype.  The  two  units  required  less  than  3  square 
feet  of  space  on  a  table  or  workbench.  The  two  units  could  be  conveniently 
located  on  a  workbench  and  leave  plenty  of  room  for  the  technician  to  work. 
By  comparison,  the  Rockwell/Hughes  CMAS  I  prototype  installed  at  Offutt  AFB 
used  approximately  50  square  feet  of  floor  space  and  required  a  rearrangement 
of  the  Radar  Shop. 
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Figure  36.  Grid  Compass  Computer,  Model  1139. 


Table  5.  CMAS  II  Hardware  Specifications 

Feature 

■T-T4  [  2  m? 

Computer: 

Model 

Grid  Compass  Model  1139 

Memory 

Random  Access  Memory 

512  KBytes 

Bubble  Memory 

384  KBytes 

CPU 

Intel  8086 

Arithmetic  Coprocessor 

Intel  8087 

Display 

Electroluminescent 

Active  Display  Area 

19.2  cm  9.6  cm 

(7.56  x  3.78  Inches) 

Resolution 

512  x  256  pixels 

(66.6  pixels  per  Inch) 

Weight 

10  pounds 

Dimensions 

11.5  x  15  x  2  Inches 

Power  Source 

120/220  VAC 

Keyboard 

Standard  QWERTY 

Disk  Drive: 

Model 

Grid  Model  2101 

Capacities 

Hard  Disk 

10  MBytes 

Floppy  Disk 

360  KBytes 

Power  Source 

120/220  VAC 

Software 


There  was  no  off-the-shelf  software  available  with  all  of  the  capabilities 
required  for  the  CMAS  II.  However,  a  software  package  was  available  from  Grid 
Systems  which  had  many  of  the  features  required.  This  software,  known  as  Grid 
Demolnterpreter,  had  been  developed  by  Grid  Systems  to  create  and  present 
"slide  shows"  to  demonstrate  the  capabilities  of  the  Grid  Compass  II 
computer.  The  Demolnterpreter  software  provided  the  capability  to  create  and 
display  a  frame  composed  of  text,  graphics  (bit-mapped),  or  a  combination  of 
text  and  graphics.  The  capability  was  provided  to  go  to  the  next  frame  in  a 
predetermined  sequence  by  pressing  a  specified  key  or  to  branch  to  one  of 
several  frames  at  the  user's  option  (by  selecting  from  a  menu).  The 
Demolnterpreter  software  provided  the  basic  capabilities  to  create  a  small 
data  base  composed  of  frames  of  information  for  presentation,  to  present  the 
frame  of  data,  and  to  move  about  in  the  data  base.  However,  analysis  of  the 
capabilities  of  the  Demolnterpreter  software  indicated  that  the  software  did 
not  provide  adequate  capability  to  move  about  in  the  data  base  for  our 
purposes.  In  addition,  the  process  required  to  create  data  for  presentation 
using  the  Demolnterpreter  was  too  slow  and  cumbersome  for  use  in  developing 
the  large  amount  of  data  required  for  the  CMAS  II.  Also,  there  was  concern 
that  the  Demolnterpreter  software  would  not  provide  a  sufficiently  rapid 
response  time. 

Although  the  Demolnterpreter  software  did  not  meet  the  requirements  for 
the  CMAS  II,  it  provided  a  good  starting  point.  Extensive  revisions  were  made 
to  the  Demolnterpreter  software  by  AFHRL  to  add  the  capabilities  required  and 
to  simplify  creation  of  data  for  presentation  on  the  system.  The  following 
capabilities  were  added  to  the  Demolnterpreter  software: 

1.  Capability  to  predefine  windows  for  text  and  graphics. 

2.  Simplified  procedures  for  inputting  textual  information  (e.g.,  word 
wrap). 

3.  Simplified  procedures  for  defining  the  position  of  text  and  graphics 
on  the  display. 

4.  Capability  to  go  directly  to  a  specific  frame  or  section  of  the  data 
base  by  inputting  an  identifier  (e.g.,  frame  number). 

5.  Capability  to  return  directly  to  a  starting  point  after  branching  to  a 
different  frame. 

6.  Simplified  procedures  for  defining  and  changing  fonts  to  be  used  in  a 
window. 

7.  Capability  to  display  graphics  more  rapidly. 

8.  Capability  to  pan  or  scroll  graphics. 

9.  Capability  to  perform  arithmetic  computations  on  data  input  by  the 
user. 

With  the  addition  of  the  capabilities  described  above,  the  Demolnterpreter 
software  provided  the  basic  capabilities  desired  for  the  CMAS  II,  with  the 
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exception  of  the  capability  to  rescale  or  zoom  graphics  and  the  capability  to 
display  a  frame  of  procedural  data  in  less  than  5  seconds.  The  addition  of  a 
suitable  zoom  capability  would  have  required  a  change  in  the  basic  approach  to 
handling  graphics  (from  bit-map  graphics  to  vector  graphics)  and  would  have 
required  an  extensive  software  development  effort.  The  time  and  resources 
required  to  develop  the  software  to  add  the  zoom  capability  were  beyond  the 
scope  of  the  effort.  Thus,  the  zoom  capability  was  not  provided  in  the  CMAS 
II. 

The  requirement  that  a  frame  of  procedural  data  be  presented  in  less  than 
5  seconds  was  considered  to  be  essential.  This  problem  was  solved  by  loading 
frequently  used  graphics  into  the  computer  memory  at  the  beginning  of  the 
session  and  by  developing  software  to  "precompile"  the  data  base  into  a  binary 
form.  This  strategy  proved  to  be  effective.  The  time  required  to  present  a 
frame  of  procedural  data  was  reduced  to  between  2  and  3  seconds  for  a  typical 
procedural  frame— well  within  the  requirement.  Although  the  complete 
presentation  of  a  frame  took  between  2  and  3  seconds,  the  response  time 
appeared  to  be  much  faster  since  the  computer  would  actually  begin  painting 
the  frame  in  less  than  .5  second. 


Man/Machine  Interface 

The  MM I  was  designed  to  provide  the  technician  with  maximum  flexibility  in 
using  the  system.  A  major  consideration  was  to  provide  the  user  with  the 
ability  to  go  from  one  location  in  the  data  base  to  any  other  location  in  the 
data  base  by  a  simple  means,  requiring  a  limited  number  of  keystrokes.  This 
capability  was  considered  essential  to  eliminate  the  "boxed-in"  feeling 
reported  by  technicians  participating  in  the  CMAS  I  field  test. 

Another  goal  in  the  MMI  design  was  to  limit  the  number  of  keystrokes 
required  for  most  routine  operations,  such  as  going  to  the  next  frame  or 
responding  to  a  "yes"  or  "no"  question.  This  capability  was  provided  by 
making  it  possible  to  go  to  the  next  frame  by  pressing  "SPACE  BAR,"  to  go  to 
the  previous  frame  by  pressing  "B,"  or  to  respond  to  a  "Yes  or  No"  question  by 
pressing  "Y"  or  "N."  Access  to  other  parts  of  the  data  base  such  as  Theory  of 
Operation  was  provided  through  an  Options  Menu  (Figure  37)  which  could  be 
recalled  by  pressing  "0."  The  optional  materials  could  be  retrieved  by 
selecting  from  the  menu.  Cues  were  presented  at  the  bottom  of  each  frame  to 
advise  the  user  of  the  available  choices. 

Direct  access  to  any  part  of  the  data  base  was  provided  by  the  direct 
access  mode.  The  direct  access  mode  could  be  entered  through  the  Table  of 
Contents  frame  or  through  an  options  menu.  Using  the  direct  access  mode,  the 
user  could  go  directly  to  any  frame  or  procedure  by  entering  the  frame  number 
or  procedure  title  and  pressing  RETURN.  A  direct  access  mode  was  also 
provided  for  location  of  parts  information.  By  entering  the  part  number  or 
reference  designator,  the  user  could  go  directly  to  the  frame  providing 
detailed  information  on  and  an  illustration  of  the  desired  part. 

Initial  entry  to  the  data  base  could  be  achieved  through  the  direct  access 
mode  (an  option  at  the  title  frame)  or  through  a  sequence  of  Table  of  Contents 
frames  which  progressively  narrowed  the  choices  until  the  desired  information 
or  procedure  was  Identified. 


OPTIONS 


1.  Theory  of  Operation 

2.  More  Detail 

3.  Direct  Access  Node 

4.  Table  of  Contents 

5.  Return  to  Last  Frane 


Enter  Number  of  Desired  Upf i 


Figure  37.  Example  Options  Menu  Frame. 


The  diagrams  presented  in  Figures  38  and  39  illustrate  the  data  access 
capabilities  provided  by  the  system. 

For  graphics  larger  than  screen  size,  a  capability  was  provided  to  scroll 
or  pan  the  graphic.  When  the  scroll  option  was  available,  an  icon  composed  of 
up/down/ left/right  arrows  was  displayed  in  the  lower  right  corner  of  the 
display.  When  the  icon  was  present,  the  user  could  scroll  the  display  up, 
down,  left,  or  right  by  using  the  arrow  keys  on  the  keyboard.  When  scroll ec, 
the  graphic  gave  the  appearance  of  moving  slowly  across  the  screen.  This 
feature  was  provided  primarily  for  the  presentation  of  schematics  and  other 
large  diagrams  (see  Figure  40). 

In  some  tasks,  the  technician  is  required  to  perform  arithmetic 
calculations  to  complete  a  test  or  measurement.  The  CMAS  II  provided  a 
feature  which  allowed  the  user  to  input  the  raw  data  into  the  computer  using 
the  keyboard.  The  computer  would  respond  by  asking  the  user  to  verify  that 
the  data  were  input  correctly.  When  confirmed  by  the  user,  the  system 
performed  the  calculations  and  displayed  the  result.  If  an  out-of -tolerance 
condition  was  found,  the  computer  advised  the  user  and  provided  an  option  to 
go  directly  to  the  appropriate  procedure  for  further  troubleshooting  or  repair 
Instructions. 

One  of  the  most  time-consuming  tasks  that  a  technician  Is  required  to  perform 
Is  the  location  of  parts  Information.  For  example.  If  a  technician  Is 
troubleshooting  a  fault  using  a  TO,  the  TO  will  tell  him  the  name  or  reference 
designator  of  the  part  that  must  be  replaced.  He  must  then  go  to  the  IPB  to 
obtain  the  part  number  and  other  essential  information.  Then,  if  he  wants  to 
order  the  part,  he  has  to  go  to  a  microfiche  file  to  obtain  the  National  Stock 
Number.  The  CMAS  II  provided  a  solution  to  this  problem.  When  a  requirement 
for  a  part  was  identified,  the  technician  could  select  Illustrated  Parts 
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38.  Flow  of  Information  -  Procedures 
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39.  Flow  of  Information  -  Troubleshooting. 


Information  from  the  Options  Menu.  The  system  would  respond  with  the 
necessary  information  on  the  part,  including  the  reference  designator;  part 
number;  stock  number;  units  per  assembly;  source  code;  repair  code;  source, 
maintainability,  and  recoverability  code;  and  a  description  (see  Figure  41). 
The  CMAS  II  also  provided  the  capability  to  recall  specific  information  on  a 
part  by  using  the  direct  access  feature.  In  this  case,  if  the  technician 
entered  the  reference  designator  or  part  number,  the  system  would  respond  with 
complete  information  on  the  part. 


Figure  40.  Example  Schematic  Diagram  Frame. 

The  CMAS  II  demonstrated  a  unique  approach  to  the  identification  of  parts 
from  an  IPB  illustration.  Information  on  specific  components  of  a  printed 
circuit  board  could  be  obtained  by  moving  an  arrow  displayed  on  the  screen  to 
the  component  of  interest.  When  the  arrow  crossed  the  boundaries  of  the 
component,  the  component  was  highlighted.  The  user  could  then  retrieve 
information  on  the  component  by  pressing  the  return  key  (see  Figure  42). 

The  CMAS  II  provided  many  opportunities  to  branch  or  otherwise  move  to  a 
different  part  of  the  data  base.  In  some  cases,  the  user  might  want  to  return 
to  his  point  of  departure  from  the  procedure  he  was  using  (e.g.,  go  to  theory 
of  operation  and  return  to  the  step  being  performed).  The  CMAS  II  permitted 
the  user  to  return  to  the  appropriate  point  by  pressing  WR." 


Data  Presentation  Formats 

Formats  were  developed  for  the  presentation  of 
data  in  two  levels  of  detail  (Track  1  -  less  detail.  Track  2 


procedural  technical 
rv  ?  -  more  detail). 


Attempts  to  prepare  data  In  three  tracks  indicated  that  a  third  track 
(more  detailed)  is  not  practical  for  use  with  this  type  of  equipment 
(electronic).  Three  tracks  may  be  appropriate  for  other  types  of  equipment. 


Formats  provided  for  support  information  were  in  only  one  level  of  detail. 
All  formats  used  a  7  x  9  pixel  character  to  present  technical  information,  and 
a  5  x  7  pixel  font  to  present  reference  information  (TO  number,  etc).  All 
formats  reserved  the  top  portion  of  the  screen  to  display  the  TO  number, 
section  title,  and  frame  number  to  provide  location  information.  The  bottom 
of  the  screen  displayed  the  options  available  from  that  frame.  The  basic 
formats  used  for  CMAS  II  are  described  below. 


12P4-2APX&4-2  ILLUSTRATED  PARTS 

Reference  Designation  -  A5A1 

Part  nunber  -  01A236533 

Stock  Nunber  -  5895-00-946-1823 

Units/Radio  -  1 

Source  Code  -  PI 

Repair  Code  -  F 

Description:  SCALER, MATRIX  I 

Press  NEXT  for  infornation 
on  individual  conponents. 


BREAKDOWN . .  >5  hSAU 


999  999  9^ 

0)1  D  I  Q 
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Figure  41.  Example  Parts  Information  Frame. 


12P4-2APX64-2  ILLUSTRATED  PARTS 

Move  arrow  to  desired  component 
with  cursor  position  keys  and 
press  RETURN. 

It  nay  be  necessary  to  press  the 
CODE  key  while  using  the  cursor 
position  keys  to  novc  the  arrow 
one  pixel  at  a  tine. 


BREAKDOWN 


999  999  91, 

0))0  )  o 

o  o  o 
0  o  o  « 

O  0  □  Dp 

‘  °  $0  Ofllffp/ 
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Figure  42.  Example  Parts  Selection  by  Graphics  Menu  Frame. 


Procedural  Data  Formats.  The  formats  developed  for  procedural  data  were 
based  upon  the  Job  6uide  concept  and  were  similar  to  those  proposed  by 
Hatterick  (1985)  and  those  used  for  the  CMAS  I  effort.  However,  in  addition 
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to  differences  in  display  size  and  shape,  the  CMAS  II  differed  from  the  CMAS  I 
formats  in  the  following  ways: 

1.  Presented  the  TO  number,  procedure  title,  and  frame  number  at  the  top 
of  the  frame.  This  information  helped  the  user  maintain  his  orientation  in 
the  data  base  and  provided  reference  information  for  later  use  with  the  direct 
access  mode. 

2.  Provided  only  those  callouts  on  a  graphic  that  were  actually 
referenced  in  the  frame.  This  eliminated  the  need  for  the  user  to  search 
through  numerous  unused  callouts  to  locate  the  referenced  callout. 

The  procedural  formats  provided  step-by-step  instructions  supported  as 
appropriate  by  locator  illustrations  showing  the  locations  of  the  referenced 
components  or  graphics  presenting  test  information  (such  as  illustrations 
showing  expected  waveforms  on  an  oscilloscope  display).  Callouts  were  used  to 
key  the  procedure  to  the  illustration.  In  most  cases,  the  technical  data 
portion  of  the  display  was  divided  into  three  windows,  as  shown  in  Figure  43. 
The  windows  were  used  in  three  basic  layouts  to  present  the  data: 

1.  Textual  information  on  the  left  (window  one);  illustrations  in  windows 
two  and  three. 

2.  Textual  information  in  windows  one  and  three;  an  illustration  in 
window  two. 

3.  Textual  information  in  all  three  windows. 


TO  Humber  Procedure  Title  Frame  Number 
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Other  text  or  graphics  windows  could  be  defined  by  the  author.  The  above 
arrangement  was  used  whenever  practical  to  maintain  consistency  in  the 
presentation  of  the  data  and  to  simplify  creation  of  the  data. 


Special  cases  existed  which  required  the  above  layouts  to  be  modified. 
One  case  was  the  handling  of  warnings,  cautions,  and  notes.  Warnings, 
cautions,  and  notes  were  displayed  one  at  a  time,  centered  vertically  and 
horizontally  on  the  screen  with  the  word  "WARNING,"  "CAUTION,"  or  "NOTE" 
highlighted  above  the  text.  This  was  done  to  ensure  that  warnings,  cautions, 
and  notes  were  not  skipped  over.  Another  special  case  was  when  a  graphic  was 
too  large  to  fit  into  the  specified  window.  There  were  several  options 
available  to  solve  this  problem.  The  first  was  to  scroll  the  graphic,  using 
the  cursor  control  keys.  For  extremely  large  drawings,  such  as  schematics, 
the  entire  screen  was  used  except  for  the  top  and  bottom  information 
sections.  When  scrolling  was  offered  as  an  option,  an  icon  consisting  of  four 
arrows  pointing  up,  down,  left  and  right  was  displayed  in  the  lower  right-hand 
corner  of  the  screen.  Another  special  case  was  when  the  drawing  was  long  and 
narrow.  In  this  case,  the  drawing  was  displayed  on  the  bottom  portion  of  the 
screen,  with  text  placed  above  the  drawing. 

As  indicated  earlier,  the  procedural  data  were  presented  in  two  tracks. 
The  formats  were  similar  for  the  two  tracks.  The  primary  difference  was  that 
the  Track  2  frames  were  always  supported  by  locator  illustrations.  Track  1 
frames  usually  did  not  include  locator  illustrations.  Examples  of  Track  1  and 
Track  2  procedural  frames  are  shown  in  Figures  44  and  45. 


12P4-2APX64-2  PRELIMINARY  CONTROL  SETTINGS  AND  CONNECTIONS 

a.  Connect  equipnent  as  shown. 

_ AN-UPM-137A 


2  1  6C 


CHAN  A 
UIDEO  IN 


EXT  SYNC 
IN 


0  TRIG 


CW  OUT 
1090  MHz 
XTAL  OR  SWEEP 


ATTACH 

.  DUMflV  load 


CHALET  AG 
UAR  AMPL 


In  DEO  AIJX  rfl.ldio  MHz  LOW| 
JUT  MOO  IN  I — OUT 


AN--APM-239A 


N/OUT 


PECE I UER-TRANSPI I TTER 


MODE  C 
ENCODER 

cf: 

TRANS¬ 

PONDER 

J2 

0 

CX-1090VAPX-239A-W1 

Figure  44.  Example  Track  1  Frame. 


The  text  in  each  frame  consisted  of  the  maximum  number  of  steps  that  could 
be  legibly  included  in  the  screen  with  the  associated  graphics.  All  steps 
were  labeled  alphabetically  within  each  complete  procedure  (e.g.,  RECEIVER 
SENSITIVITY  CHECK).  Individual  steps  were  single-spaced,  with  double-spacing 
between  steps  to  make  it  easier  for  the  user  to  keep  his  place  and  not  skip 
steps.  In  many  cases,  the  graphics  determined  the  number  of  steps  displayed 


on  a  screen.  If  two  consecutive  steps  required  the  same  graphics,  the  steps 
were  displayed  together.  However,  if  they  required  different  graphics  and 
there  was  not  enough  room  for  all  required  graphics  in  the  frame,  the  steps 
were  presented  separately. 


12P4-2APX64-2  PRELIMINARY  CONTROL  SETTINGS  AND  CONNECTIONS 

c.  Connect  RG-62A/U  cable  fron 
sis  generator  CHAL/TAG  VAR 
AflPL  (1)  to  signal  generator 
AUX  MOD  IN  (4). 


2.1.6.3C 


Connect  10-inch  RG-62A/U  cable 
(supplied  with  AN/UPM-137A ) 
fron  sisnal  generator  IN/OUT 
9. 5-50 W  (3)  to  sisnal 
generator  LOU  20dB  5B-Z000U 
(2). 


..  (  HI.  t  LhP 


full  t 


oOoy  DM  vOAft 
ocfip  \U>  jov; 

O  ®Q  O  O  O  QOo 

♦  ♦  *  ♦  ♦  ♦  ♦  •  ♦  ♦  ♦  ♦i" 


SIS  Generator 


Figure  45.  Example  Track  2  Frame. 


Input  Conditions  frame(s)  were  provided  at  the  beginning  of  each 
procedure.  Input  Conditions  frame(s)  provided  the  technician  with  information 
needed  to  prepare  for  the  job  (see  Figure  46).  The  following  information  was 
provided  for  each  task: 

Applicable  Serial  Numbers 
Personnel  Required  (number  and  type) 

Parts  Required 
Supplies 

Special  Tools  and  Test  Equipment 

Warnings,  Cautions,  and  Notes  applicable  to  the  task 

Table  of  Contents.  The  Table  of  Contents  was  displayed  in  a  menu  format. 
The  title  appeared  at  the  top,  with  the  choices  listed  below.  Each  choice  was 
numbered.  Selections  were  made  by  entering  the  number  of  the  desired  choice 
(see  Figure  47). 

Schematic  Diagrams.  A  full-screen  format  was  used  to  present  schematics 
and  other  larger-than-screen-size  diagrams  (see  Figure  40).  As  for  other 
types  of  data,  the  TO  number,  schematic  title,  and  frame  number  were  displayed 
at  the  top  of  the  frame.  User  option  information  was  presented  at  the  bottom 
of  the  frame.  An  Icon  (left/rlght/up/down)  was  displayed  in  the  lower  right 
corner  to  tell  the  user  that  the  graphic  was  larger  than  the  screen  and  could 
be  scrolled.  Although  the  CMAS  II  test  did  not  include  other  types  of 
diagrams  (such  as  hydraulic  diagrams),  the  same  format  could  be  used  for  those 
diagrams. 
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12P4-2*PX64-2  CHECKOUT  AND  ANALYSIS  2.1  1.  1C 


INPUT  CONDITIONS 

Applicable  Serial  Nos:  All 

Supplies:  None 

Personnel  Required:  One 

Special  Tools  and  Test  Equipment: 

AN/UPH-137A  Radar  Test  Set 
AN/APM-239A  Transponder  Test  Set 
AN/APfl-245  Node  4,  Signal  Generator  Sinulator 
Fault  Isolation  Neter 


*  Hr  I  -Phi.  t  L  hP  E;*-*1,  (•  -  fcr  i  F  '  {  h  >1  KF  '  N 


Figure  46.  Example  Input  Conditions  Frame. 


12P4-2APX64-2  TABLE  OF  CONTENTS 


1.  THEORY  OF  OPERATION 

2 .  CHECKOUT  AND  ANALYSIS 

3.  DIRECT  ACCESS  NODE 

4.  ILLUSTRATED  PARTS  BREAKDOWN 

5.  TROUBLESHOOTING  PROCEDURES 

6.  SCHEMATIC  DIAGRAMS 


Figure  47.  Example  Table  of  Contents  Frame. 


Theory  of  Opera cl on.  Formats  for  presentation  of  theory  of  operation 
information  varied  with  the  type  of  information  presented.  If  the  data  were 
text  only,  the  information  was  presented  in  one  column,  single-spaced,  in  one 
large  text  window.  If  a  graphic  was  required,  a  portion  of  the  frame  was 
reserved  for  the  graphic  (see  Figure  48). 


CMAS  II  Evaluation 


The  CMAS  II  was  evaluated  by  developing  technical  data  for  a  testbed 
system,  inputting  the  data  to  the  Grid  Compass  II  computer,  and  taking  it  to 
an  operational  unit  for  a  field  demonstration.  There  it  was  used  by  Air  Force 
technicians  to  perform  maintenance  on  the  testbed  system. 
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12P4-2APX64-2  UMerall  Functional  Operation  1 . 1 .  IT 

The  rmcmivmr-transnittmr  consists  of  nine  nodules: 

RF.  IF  amplifier.  decoder,  delay  line,  coder, 
reference  signal  generator,  transnit ter,  test,  and 
power  supply.  (Test  and  power  supply  are  not  shown.) 

The  inconing  pulse-coded  interrogations  received  at 
the  antenna  are  nixed  in  the  RF 
nodule  with  a  local  oscillator 
signal.  The  heterodyne 
difference  frequency  of  59.5 
negacycles  is  anplified  and 
denodulated  in  the  IF  anplifier 
nodule  and,  after  video 
processing,  is  sent  to  the 
decoder . 


Figure  48.  Example  Theory  of  Operation  Frame. 


Test  Data  Development 

The  RT-728A  transponder  from  the  AN/APX-64  Identify  Friend/Foe  ( IFF) 
transponder  was  chosen  as  the  testbed  for  use  in  evaluating  the  CMAS  II.  The 
RT-728A  was  also  used  in  the  CMAS  I  evaluation.  The  transponder  is  used  in 
the  B-52,  KC-135,  T-38,  and  many  other  aircraft.  It  is  also  used  by  the  Navy 
and  Marines  in  some  helicopters.  The  transponder  was  chosen  because  of  the 
availability  of  a  functional  RT  unit  and  adequate  technical  Information  on  the 
system.  Also,  subject-matter  experts  on  the  equipment  and  a  suitable  test 
site  were  readily  available. 

Available  resources  and  time  constraints  did  not  permit  the  development  of 
a  complete  set  of  automated  technical  data  for  the  RT-728A.  A  sufficient 
sample  of  maintenance  data  and  support  information  was  developed  to  permit  an 
evaluation  of  the  system.  Graphics  were  provided  as  necessary  to  support  the 
maintenance  procedures,  theory  of  operation,  and  IPB  Information.  The 
following  data  were  developed: 

1.  Maintenance  Procedures: 

a.  Minimum  Performance  and  Checkout  and  Analysis  procedures  for  the 
RT  Unit  (Section  6  of  the  TO). 


module). 


b.  Troubleshooting  procedures  for  the  RT  Unit  (troubleshoot  to  the 


c.  Troubleshooting  procedures  for  the  IF  Module  (limited  to 
identification  of  faults  on  one  printed  circuit  board). 

2.  Support  Data: 


a.  Theory  of  Operation. 

b.  Illustrated  Parts  Breakdown  (limited  to  basic  unit  and  IF  module). 
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c.  Schematic  Diagrams  (IF  module  printed  circuit  boards  only). 

The  original  TO  data  were  written  in  a  narrative  style  with  limited  use  of 
graphics.  Much  of  the  data  needed  was  located  in  different  sections  of  the  TO 
and  required  the  technician  to  search  for  these  data.  Thus,  to  meet  the  CMAS 
goal  of  providing  the  technician  with  all  needed  information  in  one  place,  it 
was  necessary  to  restructure  the  data.  This  required  a  reanalysis  of  the  data 
to  gather  the  necessary  information  and  to  provide  links  between  related 
portions  of  the  data  so  that  relevant  information  could  be  rapidly  retrieved. 
The  data  from  the  TO  were  completely  rewritten  to  fit  the  CMAS  II  formats. 
Where  necessary,  additional  information  was  created  and  added.  All  other 
information  in  the  data  base  that  could  reasonably  be  expected  to  be  required 
at  that  point  was  identified  and  included  with  branching  parameters  for  each 
frame  of  procedural  data. 

The  TO  provided  very  little  troubleshooting  information.  Thus,  once  a 
fault  was  identified  during  the  checkout  procedure,  the  technician  was 
expected  to  use  system  operation  information,  theory  of  operation  information, 
schematic  diagrams,  etc.,  and  his  experience,  to  develop  his  own  trouble¬ 
shooting  strategy  to  troubleshoot  the  problem.  Since  the  TO  contained  only 
limited  troubleshooting  information,  it  was  necessary  to  create 
troubleshooting  procedures  for  use  in  CMAS  II.  Assistance  in  developing  the 
troubleshooting  procedures  was  obtained  from  an  experienced  technician  at 
Offutt  AFB.  Since  available  resources  were  limited,  it  was  possible  to 
develop  only  a  limited  amount  of  troubleshooting  data.  Data  were  developed  to 
troubleshoot  the  IF  module  to  isolate  a  faulty  circuit  board  in  the  module  and 
to  isolate  a  fault  on  one  of  the  boards  (the  A5A1  board). 

Story  boards  were  created  from  the  original  TO  by  several  in-house 
personnel.  An  example  story  board  is  shown  in  Figure  49.  Each  form 
represented  one  "frame"  on  the  automated  system..  Previous  and  next  frames, 
possible  branching  requirements,  graphics  needed,  and  callout  locations  for 
the  graphics  were  Included  on  each  story  board.  The  story  boards  were  then 
transformed  into  the  Demo Interpreter  machine-executable  code  by  an  in-house 
analyst.  Once  complete,  all  data  frames  were  reviewed  for  standardization  and 
technical  accuracy.  These  data  were  then  taken  to  Offutt  AFB  for  validation 
by  maintenance  technicians  with  experience  in  maintaining  the  RT-728A. 


Field  Demonstration 


When  data  development  and  validation  were  completed,  the  CMAS  II  was  taken 
to  Grissom  AFB  for  a  2-week  field  demonstration.  The  data  base  for  the  CMAS 
II  was  too  small  to  permit  a  full-scale  evaluation  of  the  system.  Thus,  the 
goals  of  the  field  demonstration  were  somewhat  limited.  The  objectives  of  the 
CMAS  II  field  demonstration  were  to: 

1.  Demonstrate  the  use  of  an  automated  technical  data  presentation  system 
In  an  operational  environment. 

2.  Evaluate  the  usability  of  the  system  (including  MMI  techniques,  data 
access  techniques,  and  data  formats). 
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Figure  49.  Example  Story  Board 


3.  Determine  the  overall  acceptance  of  the  system  by  technicians  and 

identify  features  which  contributed  positively  and  negatively  to  user 
acceptance. 

4.  Develop  preliminary  indications  of  the  effectiveness  of  the  system  as 
a  data  presentation  system  in  comparison  to  the  existing  paper-based  system. 

5.  Identify  methods  to  improve  the  system. 

Field  Demonstration  Procedures.  CMAS  II  was  placed  in  an  intermediate 

level  shop  (Radar)  at  Grissom  AFB  for  a  2-week  period  in  August  1985.  The 

radar  shop  had  the  responsibility  for  maintaining  the  AN/APX-64  and  had 

personnel  assigned  who  had  a  wide  range  of  experience  in  maintaining  the 

system.  The  demonstration  was  conducted  on  a  noninterference  with  normal  shop 

operations  basis.  In  some  instances,  it  was  necessary  for  some  technicians  to 
interrupt  their  participation  in  the  field  demonstration  to  perform  mission 
essential  tasks.  The  short  time  period  and  the  noninterference  policy  limited 
the  number  and  experience  levels  of  the  technicians  participating  in  the 

demonstration.  Other  limiting  factors  were  the  small  amount  of  data  in  the 
system  and  the  number  of  planted  "faults"  which  could  be  placed  in  the 

receiver-transmitter  unit. 

Eight  technicians  from  the  Radar  Shop  served  as  subjects  for  the  study. 
Their  experience  in  maintaining  the  AN/APX-64  ranged  from  6  months  to  12 
years.  Half  of  the  subjects  had  more  than  1  year  of  experience  in  maintaining 
the  system,  and  half  had  less  than  1  year  of  experience  on  the  system.  The 
subjects  included  a  quality  assurance  inspector  and  a  Field  Training 
Detachment  instructor  (who  was  currently  teaching  maintenance  of  the  system). 
In  addition,  one  technician  from  the  Inertial  Navigation  Shop  was  tested. 
This  individual  had  no  training  or  experience  on  the  system.  He  was  Included 
to  provide  an  indication  of  the  ability  of  the  system  and  the  reformatted  data 
to  support  performance  by  very  inexperienced  personnel. 

Each  technician  was  given  5  minutes  of  instruction  on  the  use  of  CMAS  II 
and  given  a  set  of  written  instructions  (Appendix  A)  on  how  to  use  the  system. 
At  the  end  of  this  session,  each  technician  completed  an  exercise  to 
!  demonstrate  his  ability  to  use  the  CMAS  II. 

After  the  technicians  were  trained  to  use  the  CMAS  II,  they  were  asked  to 
perform  three  tasks  on  the  RT  unit:  two  checkout  tasks  and  one 
troubleshooting  task.  The  two  checkout  tasks  were  judged  to  be  equal  in 
difficulty.  Both  checkout  tasks  required  setting  up  the  test  equipment  and 
performing  the  checkout  procedure  until  an  out-of -tolerance  condition 

(sensitivity  adjustment)  was  identified.  When  the  out-of-tolerance  condition 
was  identified,  the  technician  was  Instructed  to  adjust  the  sensitivity  until 
it  was  within  tolerance.  One  checkout  task  was  performed  using  the  CMAS  II, 

I  and  one  was  performed  using  the  TO.  The  order  of  presentation  was 

j  counterbalanced  so  that  half  of  the  subjects  performed  the  first  task  using 

i  the  CMAS  II,  and  half  used  the  TO  for  the  first  task. 

i 

• 

|  The  troubleshooting  task  required  Identifying  an  out-of-tolerance 

,  condition  and  then  troubleshooting  the  system  to  identify  the  faulty  module 

!  and  the  faulty  component  within  the  module  (to  the  piece-part  level).  The 

[  fault  for  this  task  was  inserted  in  a  printed  circuit  board  by  damaging  the 
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card  to  interrupt  the  signal  flow.  The  damage  was  detectable  only  upon 
careful  observation.  For  the  troubleshooting  task,  the  technician  was  started 
at  a  point  further  into  the  checkout  procedure  and  instructed  to  perform  the 
checkout  until  an  out-of-tolerance  condition  was  found  and  then  isolate  the 
cause  of  the  problem.  Although  technicians  in  the  intermediate  level  shop 
were  normally  authorized  to  troubleshoot  to  the  card  level  only,  they  were 
instructed  to  isolate  the  problem  to  the  piece  part.  Isolation  to  the 
piece-part  level  was  included  to  further  test  the  limits  of  the  system  to 
support  troubleshooting.  The  troubleshooting  task  was  performed  using  the 
CMAS  II. 

The  performance  of  the  technicians  was  evaluated  by  an  observer  who 
recorded  whether  the  problem  was  successfully  completed,  the  time  to  complete, 
errors  made,  and  any  problems  encountered.  After  the  performance  tests  were 
completed,  the  technicians  performed  a  series  of  exercises  designed  to 
evaluate  whether  the  technique  used  to  present  schematic  diagrams  was 
adequate.  The  exercises  consisted  of  having  the  technician  locate  specific 
components  on  the  diagram,  follow  signal  flows  on  the  schematic,  and  identify 
sources  of  inputs  and  outputs. 

Following  the  performance  tests  and  schematics  exercises,  each  subject 
completed  a  questionnaire  and  was  interviewed  to  determine  what  he  liked  and 
disliked  about  the  system  and  to  solicit  recommendations  for  improvement. 

Field  Demonstration  Results.  The  times  required  for  the  technicians  to 
complete  each  performance  test  are  shown  in  Table  6.  As  illustrated  in  the 
table,  all  of  the  technicians  were  able  to  satisfactorily  complete  all  of  the 
tests.  Comparison  of  the  data  for  performance  using  the  TO  and  CMAS  II  for 
the  checkout  procedures  reveals  that  the  times  were  essentially  the  same  for 
both  conditions.  No  comparison  is  possible  for  troubleshooting  since  the  TO 
condition  was  not  tested.  The  performance  times  should  be  viewed  as  general 
indicators  of  performance  times.  There  were  enough  interferences  and 
extenuating  circumstances  in  the  data  collection  process  that  the  data  cannot 
be  considered  sufficiently  clean  for  a  formal  statistical  analysis. 

The  questionnaires  administered  and  interviews  conducted  after  the 
completion  of  the  performance  test  yielded  many  very  positive  comments  and 
only  a  few  negative  comments.  The  most  valuable  information  was  obtained  from 
the  interviews.  The  interview  comments  are  summarized  in  Appendix  B. 

In  general,  the  technicians  were  very  receptive  to  the  system.  All  eight 
technicians  who  completed  the  questionnaire  indicated  that  they  preferred  X(T~ 
use  the  computer  over  the  paper  TO.  Most  expressed  a  desire  to  see  a  similar 
system  implemented  for  Air  Force-wide  use.  Several  expressed  disappointment 
at  the  realization  that  it  will  be  several  years  before  such  a  system  can  be 
implemented  for  widespread  use  throughout  the  Air  Force.  Positive  reactions 
expressed  Included  the  following  observations: 

1.  The  computer  response  time  was  considered  to  be  very  good  by  the 
technicians. 

2.  With  one  exception,  the  graphics  were  considered  to  be  at  least  as 
good  as  the  graphics  provided  in  the  TO. 
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3.  The  system  was  considered  to  be  very  easy  to  use. 


4.  The  procedures  provided  for  locating  and  accessing  data  were 
considered  to  be  effective.  The  direct  access  capability  was  seen  as  a 
desirable  feature. 

5.  The  presentation  of  IPB  information  was  considered  to  be  a  valuable 
feature.  The  IPB  information  presentation  capability  was  seen  as  a  time  saver. 

6.  The  computer  forced  the  technician  to  read  every  step.  This  made  it 
difficult  to  miss  a  step.  Thus,  they  felt  that  the  use  of  the  computer 
lessened  the  chance  of  making  a  mistake. 

7.  The  troubleshooting  capability  of  the  system  was  seen  as  a  valuable 
tool. 


Table  6.  Task  Performance  Times  for  CMAS  II 
Field  Test  (In  Minutes) 


Subject 

CO/CMAS  II 

CO/TO 

TS/CMAS 

H- 1 

. sr —  . 

28 

5$ 

H-2 

32 

22 

17 

H-3 

23 

25 

23 

H-4 

30 

33 

26 

Total 

137 

108 

119 

Mean 

27.00 

34.25 

29.75 

L-l 

34 

43 

no 

L-2 

52 

58 

44 

L-3 

65 

77 

266 

L-4 

31 

34 

33 

Total 

182 

212 

343 

Mean 

53 

45.50 

114.33 

Grand 

total 

319 

320 

462 

Grand 

mean 

40.00 

39.88 

_ §L£2- 

The  primary  criticism  of  the  system  expressed  by  the  technicians  concerned 
the  presentation  of  schematics.  Although  they  could  use  the  schematics  as 
presented  on  the  system,  several  felt  handicapped  by  the  fact  that  they  could 
not  see  the  whole  diagram  at  one  time.  Some  also  noted  a  problem  In  locating 
Information  along  the  edges  of  the  schematic.  Other  concerns  expressed  by  the 
technicians  included  the  following: 


1.  It  was  noted  that  when  the  direct  access  mode  was  used,  critical 
Warnings,  Cautions,  and  Notes  could  be  bypassed.  It  was  suggested  that  the 
direct  access  technique  be  modified  to  always  present  the  relevant  Warnings, 
Cautions,  and  Notes  before  presenting  the  procedural  information. 

2.  One  technician  (the  Quality  Assurance  Inspector)  questioned  the 
two-track  concept.  He  was  concerned  that  the  experienced  technician  may  not 
know  the  procedure  as  well  as  he  thinks,  and  may  miss  a  critical  step  or  cue. 

Surprisingly,  the  technicians  had  very  few  suggestions  for  improving  the 
system.  Suggestions  made  which  should  be  considered  for  the  development  of  an 
operational  system  included: 

1.  Put  the  computer  on  a  cart  similar  to  those  used  for  oscilloscopes. 
This  would  provide  more  flexibility  and  eliminate  the  risk  of  knocking  the 
computer  off  of  the  workbench.  It  would  also  make  more  work  space  available 
on  the  bench. 

2.  Use  a  larger  display.  This  would  make  it  easier  to  present  schematics. 

3.  Link  related  schematics  together  so  that  the  computer  goes  directly  to 
the  next  schematic  when  the  edge  of  the  schematic  is  reached. 

4.  Modify  the  direct  access  feature  to  present  any  relevant  Warnings, 
Cautions,  and  Notes  before  any  procedural  Information  is  presented. 


Specification  Development 

Following  the  successful  demonstration  of  the  CMAS  II,  draft  military 
specifications  for  the  content  and  format  of  automated  technical  data  were 
developed  under  contract  by  Applied  Science  Associates,  Inc.  (Evans  1986a). 
The  specification  provides  requirements  for  the  content  of  the  data, 
requirements  for  writing  the  data,  requirements  for  formatting  the  data,  and 
requirements  for  establishing  interactions  (branching,  etc.)  for  the  data. 
Format  and  presentation  requirements  are  based  primarily  upon  the  CMAS  II  and 
describe  a  very  similar  system.  The  technical  content  requirements  of  the 
specifications  are  based  primarily  on  earlier  AFHRL  job  performance  aids 
research  (Joyce  et  al.,  1973). 

In  addition  to  the  draft  content  specification,  a  draft  specification  was 
developed  establishing  system  requirements  for  an  automated  technical  data 
system  (Evans,  1986b).  The  specification  provides  system  requirements  such  as 
screen  resolution  and  system  response  times. 


CHAS  II  Refinement 


One  of  the  goals  of  the  field  demonstration  was  to  identify  ways  of  making 
the  system  better.  One  refinement  was  identified  which  could  be  readily 
Incorporated  Into  the  system.  It  was  suggested  that  the  retrieval  of  optional 
Information  could  be  simplified  by  using  a  function  key  approach  rather  than 
the  options  menu  approach.  With  the  function  key  approach,  the  options  would 
be  identified  on  the  bottom  of  the  screen  and  keyed  to  a  corresponding 
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function  key.  The  desired  option  could  then  be  selected  by  pressing  the 
corresponding  function  key.  This  approach  would  allow  the  technician  to 
select  optional  information  or  another  mode  of  operation  (direct  access)  with 
one  key  press,  rather  than  the  two  or  more  key  presses  required  for  the 
options  menu  approach. 

Since  the  Grid  Compass  II  does  not  provide  dedicated  function  keys,  it  was 
necessary  to  simulate  the  use  of  function  keys.  This  was  accomplished  by 
designating  the  numeric  keys  as  function  keys.  The  display  formats  were  then 
modified  to  provide  a  listing  of  available  options  across  the  bottom  of  each 
frame.  Each  option  was  identified  by  a  number.  The  user  could  make  a 
selection  by  pressing  the  appropriate  numeric  key.  Figure  50  presents  an 
example  of  the  revised  format  for  procedural  data. 


Figure  50.  Example  Upgraded  Format  for  Procedural  Frame. 


Other  suggestions  were  made  for  improving  the  system.  These  suggestions 
primarily  involved  ways  to  make  the  schematic  diagrams  more  usable.  However, 
the  time  and  costs  to  implement  the  suggested  approaches  were  greater  than  the 
resources  available.  Also,  in  some  cases,  the  hardware  and  software 
requirements  were  beyond  the  capabilities  of  the  Grid  Compass  II. 


Air  Force/Navv  Evaluation 

The  Navy  Personnel  Research  and  Development  Center  (NPRDC)  also  has  a 
program  to  study  methods  for  automating  technical  data  for  maintenance.  Since 
the  NPRDC  program  Is  concerned  with  many  of  the  same  problems  and  research 
concerns  as  the  AFffiL  program,  close  coordination  Is  maintained  with  NPRDC 


personnel.  When  NPRDC  personnel  visited  the  CMAS  II  field  demonstration  at 
Grissom  AFB  to  observe,  it  Mas  noted  that  the  Navy  also  has  aircraft  which  use 
the  AN/APX-64.  Subsequent  discussions  led  to  an  agreement  for  NPROC  to 
conduct  an  Independent  study  to  further  evaluate  the  CMAS  II.  It  was  agreed 
that  NPROC  would  have  full  responsibility  for  conducting  the  study  and  that 
AFHRL  would  provide  the  CMAS  II  hardware,  software,  data  base,  and  technical 
assistance  for  the  effort.  The  results  of  the  NPRDC  study  are  reported  in 
Nugent,  Sander,  Johnson,  and  Smillie  (1987).  The  following  materials  were 
adapted  from  that  report. 


Objective 

The  objective  of  the  NPRDC  study  (Nugent  et  al.,  1987)  was  to  "...extend 
the  results  of  an  earlier  Air  Force  study  and  compare  the  troubleshooting 
performance  of  military  technicians  who  obtained  information  from  conven¬ 
tional,  paper-based  maintenance  manuals  and  from  electronic  devices"  (p.  1). 


Approach 

The  experimental  approach  was  designed  to  test  the  following  hypotheses: 

1.  Troubleshooting  will  take  less  time  when  using  the  electronic 
presentation  system  than  using  technical  manuals. 

2.  Fewer  tests  will  be  performed  using  the  electronic  presentation 
system  than  using  technical  manuals. 

3.  Fewer  unnecessary  replacements  (modules  and  circuit  cards)  will  be 
made  using  the  electronic  presentation  system  than  using  technical  manuals. 

4.  More  faults  will  be  Isolated  successfully  using  the  electronic 
presentation  system  than  using  technical  manuals. 

5.  Inexperienced  technicians  using  the  electronic  presentation  system 
will  troubleshoot  as  well  as  experienced  technicians  using  technical  manuals. 

6.  When  using  the  paper-based  technical  manual,  experienced  technicians 
will  troubleshoot  better  than  Inexperienced  technicians. 

7.  Performance  differences  between  the  two  presentation  methods  will  be 
greater  for  Inexperienced  than  for  experienced  technicians. 

The  basic  approach  adopted  for  the  study  was  to  use  the  same  testbed 
system  (RT-728A  Receiver-Transmitter  of  the  AN/APX-64,  Identify  Friend  or  Foe 
System)  used  In  the  AFHRL  demonstration.  The  data  base  was  expanded  to 
provide  sufficient  data  to  support  troubleshooting  a  larger  sample  of  faults. 
Four  separate  RT-728A  failures  were  used  as  troubleshooting  problems  for  the 
effort.  One  of  the  faults  was  the  same  as  that  used  In  the  AFHRL  study. 
Precautions  were  taken  to  ensure  that  the  faults  could  not  be  identified  by 
visual  Inspection  alone  and  that  they  provided  consistent  symptom  patterns. 
Care  was  also  taken  to  ensure  that  both  the  paper  manual  and  the  automated 
system  provided  sufficient  Information  to  Isolate  the  faults.  Additions  to 
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the  data  base  included  additional  schematic  diagrams,  illustrated  parts 
breakdown  information,  and  troubleshooting  procedures  to  cover  the  four 
problems. 

Thirty-six  Navy,  Marine,  and  Air  Force  avionics  technicians  from  four 
California  military  installations  served  as  subjects  for  the  study.  The 
technicians  (12  from  each  service)  were  classified  into  two  groups: 
experienced  and  inexperienced.  Group  assignments  were  based  upon  relevant 
field  experience  (1  or  more  years  for  the  experienced  group  and  less  than  1 
year  for  the  inexperienced  group)  and  judgments  of  the  technicians' 
capabilities  by  the  immediate  supervisor.  All  subjects  were  volunteers. 
Procedures  were  implemented  to  ensure  subjects'  anonymity. 

The  subjects  were  tested  individually  at  a  standard  AN/APX-64  workbench. 
They  were  given  an  orientation  to  the  testing  procedures  and  assigned  to  one 
of  six  predetermined  sequence  orders  which  counterbalanced  the  presentation  of 
the  troubleshooting  problems  and  information  delivery  device.  They  were  then 
given  Instruction  on  the  use  of  the  CMAS  II  and  provided  an  opportunity  to 
practice  using  It. 

Each  troubleshooting  problem  was  initiated  by  giving  the  technician  a 
description  of  the  symptom  associated  with  the  failure  Inserted  in  the  RT 
unit.  The  subject  was  given  a  specific  start  point  in  the  TO  or  computer  data 
base  from  which  he  was  to  initiate  the  troubleshooting.  He  was  Instructed 
that  the  only  information  the  experimenter  would  provide  was  the  time 
available  for  the  test,  whether  a  recommended  replacement  of  a  module  or 
circuit  card  would  correct  the  failure,  or  how  to  resolve  difficulties 
encountered  In  using  the  CMAS  II.  A  1-hour  time  limit  was  allowed  for 
Isolating  the  fault  to  the  printed  circuit  card.  If  this  criterion  was  met, 
an  additional  15  minutes  were  allowed  to  Isolate  to  the  component  level; 
otherwise,  the  test  was  terminated.  The  test  was  terminated  If  the  fault  had 
not  been  isolated  prior  to  the  expiration  of  the  time  limit.  Performance  was 
observed  by  a  trained  observer  who  recorded  start/stop  times,  options  used  In 
the  data  base,  the  troubleshooting  path  followed,  and  any  other  meaningful 
observations. 

Following  the  testing  session,  the  purpose  of  the  study  and  each  subject's 
performance  on  the  problems  was  reviewed  with  each  subject.  The  subjects  were 
also  asked  to  complete  questionnaires,  and  structured  Interviews  were 
conducted. 


Results 


A  2  x  2  multiple  analysis  of  variance  (MANOVA)  design  was  used  to  analyze 
the  results. 

Nine  of  the  36  subjects  were  unable  to  Isolate  the  fault  on  one  or  both 
problems  performed  using  the  technical  manual.  Thus,  data  were  not  available 
for  all  subjects  on  the  Total  Time  to  Solution  and  Total  Tests  Performed 
measures.  As  a  result,  two  separate  analyses  were  made:  one  analysis  using 
data  for  all  36  subjects  but  omitting  the  time  and  tests-used  variables,  and 
one  analysis  using  data  for  27  subjects  for  all  variables. 
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Performance  Test  Results.  The  results  for  the  overall  sample  (N  *  36)  are 
shown  In  Table  As  may  be  seen  from  the  overall  means  (for  experienced  and 
Inexperienced  combined),  time  to  repair  and  false  replacements  were 
significantly  lower  (£  <  .01)  when  the  technicians  used  the  computer  than  when 
they  used  the  paper  technical  manual.  In  addition,  significantly  more 
problems  were  successfully  solved  (£  <  .01)  when  the  computer  was  used  than 
when  the  paper  manual  was  used.  The  table  also  indicates  that  significantly 
fewer  test  points  were  checked  using  the  paper  manual  than  using  the  computer. 


Table  7.  Performance  Differences  Resulting  from  the  Use 
of  Technical  Manuals  Versus  the  Use  of  the  Electronic 
Presentation  System  for  the  Overall  Sample  (H  *  36) * 


Measure 

Mean 

Technical 

Manual 

FM.68) 

P 

Time  to  faulted 
card  (min) 

56.5 

24.4 

35.72 

.01 

Test  points  checked 

3.6 

5.6 

12.90 

.01 

False  replacements 

1.2 

0.0 

25.96 

.01 

Problems  solved 

1.7 

2.0 

9.90 

.01 

rom  Nugent  et  al.,  1987,  p.  8. 


Similar  results  were  obtained  when  the  data  were  analyzed  for  the 
restricted  sample  (N  ■  27).  The  results  of  this  analysis  are  presented  In 
Table  8. 


Table  8.  Performance  Differences  Resulting  from  the  Use 
of  technical  Manuals  Versus  the  Use  of  the  Electronic 
Presentation  System  for  the  Reduced  Sample  (N  *  27) a 


Measure 

Mean 

technical 

manual 

Mean 

computer 

FTL681 

_ [> 

time  to  faulted  card  (min) 

45. 5 

'  2?.4 

Ki  JtiMH 

loT 

Time  to  faulted  component  (min) 

28.5 

22.6 

3.99 

.06 

Total  time  to  solution  (min) 

74.1 

.01 

Test  points  checked 

3.5 

13.31 

.01 

Components  checked 

8.8 

11.5 

3.86 

.06 

Total  tests  performed 

12.3 

16.5 

8.34 

.01 

False  replacements 

11.79 

.01 

aErom Nugent  et  al.,  1967,  p.  8. 


Questionnaire  Results.  The  results  of  the  analysis  of  the  responses  to 
the  jT  items  in  the  posttest  questionnaire  are  summarized  In  Table  9.  For 
analysis  purposes,  the  questions  were  grouped  into  four  areas:  physical. 
Information  presentation,  efficiency,  and  effectiveness.  A  mean  response  for 
each  category  was  then  computed.  The  system  was  consistently  rated  "highly 
satisfactory"  to  "outstanding”  In  the  physical  category.  The  only  exception 
was  the  adequacy  of  the  screen  size,  which  was  rated  as  "satisfactory"  (mean  * 
3.2).  The  adequacy  of  the  technical  information  presented  and  access  to  that 
information  was  rated  very  high  (mean  s  4.22).  The  CMAS  II  was  perceived  as 
being  more  effective  and  more  efficient  than  the  paper  technical  manual. 


Table  9.  Sundry  of  User  Questionnaire  Evaluation8 


Feature 

Questionnaire 

Items 

wmm 

Physical 

1  -  16 

3.95b 

Information  presentation 

17  -  24 

4.22b 

Efficiency 

25  -  27 

1.68C* 

Effectiveness 

28  -  31 

<-38b 

“Adapted  from  Nugent  et  a!.,  1987,  p.  9. 

bScale  Values:  1  *  unsatisfactory,  5  *  outstanding. 


cSca1e  Values:  1  ■  significantly  less,  5  ■  signifi¬ 
cantly  more. 

*The  low  mean  for  Items  25  -  27  Is  a  positive  response. 


Structured  Interview  Results.  The  responses  of  the  36  subjects  In  the 
structured  Interviews  also  reflected  very  positive  attitudes  toward  the  CMAS 
II.  The  responses  are  summarized  in  Table  10.  Data  are  not  provided  for 
Interview  Items  1,  2,  8,  and  9,  since  few.  If  any,  responses  were  received  to 
those  questions.  Responses  of  all  participants  were  combined  for  the 
analysis,  since  no  differences  were  noted  among  the  members  of  the  three 
services  or  between  experience  levels. 


Table  10.  Summary  of  Technicians'  Interview  Responses8 


- n«« - 

Hi!  i-iii  iwmmmmmmmmm 

n 

Percent 

#3  Level  of  detail 

a.  Switching  ease 

Yes 

31 

86 

(between  levels) 

b.  Two  levels  sufficient 

Yes 

34 

94 

c.  Performance 

Less  detail 

9 

25 

More  detail 

14 

39 

#5  Like  about  Grid 

Quick  easy  access  to 
Information 

15 

42 

Procedural 1  zed:  easy  to  go 
from  one  point  to  another 

12 

33 

Deletes  excessive  narrative; 
direct  path  to  fault 

9 

25 

Reduces  troubleshooting 
time 

8 

22 

Ease  In  switching  among 
frames 

7 

19 

#6  Not  like  about  Grid 

Cannot  see  entire  schematic 
on  the  screen 

17 

47 

#7  Mode  preference 

Grid 

27 

75 

Paper 

1 

3 

_ Both _ 

7 

19 

Analysis  of  the  interview  comments  yielded  the  following  observations: 


1.  The  multiple  levels  of  detail  feature  was  a  highly  valued  feature  of 
the  automated  presentation  system.  Experienced  and  inexperienced  technicians 
generally  showed  no  difference  in  their  preference  for  a  more-  or  less- 
detailed  presentation.  The  less-detailed  presentation  was  seen  as  more 
appropriate  for  presenting  simple  procedures  and  for  troubleshooting.  The 
more-detailed  procedures  were  seen  as  appropriate  for  presenting  extra 
information  for  inexperienced  personnel  and  for  providing  parts  information. 

2.  The  technicians  generally  felt  that  the  computer  system  was  more 
effective  in  presenting  Information  for  troubleshooting  than  the  paper 
system.  The  primary  dislike  was  the  presentation  of  schematics.  This 
complaint  was  primarily  due  to  the  Inability  of  the  system  to  present  the 
complete  schematic  for  viewing  at  one  time. 

3.  The  computer  was  preferred  over  the  paper  technical  data  by  7SX  of  the 
participants;  19X  percent  Indicated  they  would  prefer  having  both  systems. 
The  latter  preference  was  based  upon  the  rationale  that  the  paper  manual  would 
be  used  to  present  schematics  and  the  computer  would  be  used  to  provide 
instructions  for  corrective  maintenance.  Also.  3%  of  the  technicians 
Indicated  that  they  felt  theory  of  operation  and  functional  descriptions  were 
more  adequately  presented  by  the  paper  technical  data. 

Conclusions 

Based  upon  the  results  of  the  study,  the  authors  (Nugent  et  al.,  1987.  p. 
12)  concluded: 

...the  results  demonstrate  that  computers  can  be  used  as  an 
effective  means  to  present  technical  Information  In  an 
electronic  presentation  format.  If  the  technical  Information 
data  base  is  constructed  for  ease  of  access,  as  was  the 
RT-728A/APX-64(V),  maintenance  performance  in  terms  of  less  time 
and  errors  should  Improve.  More  Importantly,  technicians  appear 
willing  to  change  to  a  different  delivery  method  for  obtaining 
maintenance  information. 

Since  the  data  base  was  very  limited  in  the  present  effort, 
future  evaluations  need  to  address  extremely  large  technical 
information  data  bases  that  offer  many  ways  of  entering  and 
branching  to  the  various  types  of  Information  within  those  data 
bases. 

Discussion 

The  CMAS  II  effort  demonstrated  that  an  automated  system  for  presenting 
technical  data  is  feasible  for  Intermediate  level  maintenance.  In  addition. 
It  demonstrated  that  such  a  system.  If  properly  designed,  will  be  accepted  and 
used  by  technicians.  In  both  the  Grissom  AFB  demonstration  and  the  NPRDC 
study,  technicians  were  able  to  perform  maintenance  tasks  at  least  as 
effectively  (and  in  some  cases  more  effectively)  when  using  the  automated 
system  as  when  using  the  paper  technical  data.  In  both  cases,  the  technicians 
Indicated  a  preference  for  the  automated  system. 
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The  CMAS  II  is  only  a  prototype  system.  However,  it  does  provide  the 
basis  for  a  number  of  recommendations  for  the  development  of  automated 
technical  data  systems  for  operational  use.  These  recommendations  are 
discussed  in  detail  in  Section  VII.  Although  the  prototype  proved  to  be  an 
effective  system,  there  are  a  number  of  design  issues  which  require  further 
research.  These  issues  are  also  discussed  in  Section  VII. 


VI.  PORTABLE  COMPUTER-BASED  MAINTENANCE  AIDS  SYSTEM 

The  overall  objective  of  the  computer-based  maintenance  aids  project  has 
always  been  to  develop  a  system  that  is  suitable  to  support  both  flightline 
and  intermediate  level  maintenance.  The  earlier  CMAS  I  and  II  efforts  were 
all  directed  toward  the  development  of  a  system  for  intermediate  level 
maintenance.  The  Portable  Computer-based  Maintenance  Aids  System  (PCMAS) 
effort  was  established  to  extend  the  technology  for  flightline  maintenance  and 
to  develop  the  requirements  for  an  operational  automated  technical  data 
presentation  system. 

Concurrent  with  the  development  of  the  CMAS  I  and  CMAS  II  studies,  work  by 
the  Laboratory  was  in  progress  in  three  other  areas  which  led  to  a  broadening 
of  the  goals  of  the  PCMAS  project  beyond  the  presentation  of  technical  data. 
The  areas  were:  (a)  aircraft  battle  damage  assessment  (ABDA),  (b)  automated 
diagnostics  for  on-aircraft  maintenance,  and  (c)  Integration  of  maintenance 
information  systems.  The  areas  of  research  and  their  Impact  on  the  PCMAS  are 
described  briefly  below. 

The  ability  of  the  Air  Force  to  support  sustained  combat  operations  will 
depend  to  a  large  degree  on  Its  ability  to  rapidly  repair  battle-damaged 
aircraft  and  return  them  to  operational  status.  Perhaps  the  most  difficult 
aspect  of  the  aircraft  battle  damage  repair  (ABDR)  process  is  the  assessment 
of  the  damage.  It  has  been  proposed  that  an  automated  technical  data 
presentation  system  capable  of  providing  special  ABDA  technical  data  could 
make  a  significant  contribution  to  the  assessment  process.  Such  an  aid  would 
provide  the  assessor  with  information  which  presently  Is  not  readily  available 
(e.g.,  times  required  for  typical  repairs.  Identification  of  mission  critical 
systems,  data  on  integration  of  systems,  location  of  critical  structures).  In 
addition,  it  could  provide  rapid  access  to  Information  that  is  presently 
available  from  TOs  but  is  difficult  to  locate.  (The  information  required  to 
assess  damage  to  one  portion  of  the  aircraft  may  be  spread  throughout  the 
entire  set  of  TOs  for  the  aircraft,  requiring  hours  of  searching  to  locate.) 
It  was  also  suggested  that  such  an  aid  may  speed  up  the  assessment  process  by 
providing  the  assessor  with  detailed  "peel-away  graphics"  which  would  allow 
him  to  determine  If  there  are  critical  systems  or  structures  located  in, 
around,  or  behind  the  damaged  area  without  removing  panels  or  components. 

Work  was  in  progress  to  develop  a  joint  project  with  the  Air  Force  Flight 
Dynamics  Aircraft  Battle  Damage  Repair  Program  Office  to  develop  an  automated 
ABDA  aid  when  it  was  realized  that  the  necessary  capabilities  could  be 
provided  by  the  PCMAS  (which  was  also  In  concept  development).  Therefore,  an 
agreement  was  reached  with  the  Flight  Dynamics  Laboratory  to  evaluate  the 
PCMAS  as  an  ABDA  aid.  The  agreement  provided  for  AFHRL  to  manage  the  effort 
and  for  the  Flight  Dynamics  Laboratory  to  provide  technical  support  and 
partial  funding  for  the  effort. 
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For  several  years,  the  Laboratory  has  had  a  project  in  progress  to  develop 
automated  fault  diagnostic  techniques  for  on-aircraft  maintenance.  The 
approach  developed  calls  for  the  use  of  a  small,  ruggedized,  portable  computer 
to  connect  directly  to  the  1553  data  bus  of  the  aircraft  to  interrogate 
aircraft  systems  and  retrieve  test  and  performance  data.  The  portable 
computer  would  then  use  the  aircraft  data  with  automated  diagnostic  routines 
to  diagnose  faults  in  the  aircraft's  systems.  The  technology  has  progressed 
to  the  point  that  it  is  ready  for  testing.  During  the  development  of 
requirements  for  the  PCMAS,  it  was  recognized  that  the  PCMAS  (as  then 
envisioned)  would  have  the  capabilities  required  for  the  diagnostics 
evaluation,  with  the  exception  of  the  ability  to  connect  to  and  interact  with 
the  aircraft  data  bus.  The  addition  of  this  capability  to  the  PCMAS  was 
deemed  feasible.  Therefore,  a  decision  was  made  to  include  a  requirement  for 
a  1553  bus  interface  capability  in  the  PCMAS  and  to  use  the  PCMAS  for  the 
planned  diagnostics  evaluation. 

A  long-range  Laboratory  project  is  developing  the  Integrated  Maintenance 
Information  System  (IMIS).  The  IMIS*  will  provide  a  common  interface  to  a 
number  of  Information  systems  used  in  maintenance.  The  systems  to  be  covered 
include:  (a)  the  Automated  Technical  Order  System  (ATOS),  (b)  the  Core 
Automated  Maintenance  System  (CAMS),  (c)  supply  systems,  and  (d)  automated 
training  systems.  The  IMIS  program  is  presently  in  the  concept  development 
stage.  Studies  are  planned  to  examine  several  issues  and  possible  technical 
approaches  to  be  incorporated  In  the  IMIS.  A  small,  rugged,  portable  computer 
is  required  for  this  purpose.  The  PCMAS  has  the  capabilities  required  for 
many  of  the  proposed  studies.  Requirements  of  the  IMIS  program  have  been 
considered  in  developing  the  PCMAS  to  ensure  that  the  required  versatility  is 
provided. 


Program  6oals 

The  PCMAS  program  had  the  following  goals: 

1.  Develop  a  small,  rugged  portable  computer  with  the  capability  to: 

a.  Present  automated  technical  data  for  use  on  the  flightline; 

b.  Present  specialized  ABDA  technical  data; 

c.  Interact  with  the  aircraft  MIL-STD-1 553  data  bus,  extract  data 
from  the  bus,  and  run  automated  diagnostic  routines;  and 

d.  Serve  as  a  testbed  for  studies  in  support  of  the  development  of 
the  IMIS. 

2.  Adapt  technical  data  presentation  techniques  developed  for 
Intermediate  level  maintenance  to  present  technical  data  for  on-aircraft 
maintenance. 

3.  Develop  MMI  techniques  for  use  with  a  portable  computer  to  present 
technical  data,  ABDA  data,  and  diagnostics  data  for  on-equipment  maintenance. 

4.  Evaluate  the  PCMAS  as  an  automated  technical  data  presentation  system. 
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5.  Evaluate  the  PCMAS  as  an  ABOA  aid. 

6.  Provide  PCMAS  units  for  use  in  the  diagnostics  and  IMIS  studies. 


Development  of  System  Requirements 

Both  in-house  and  contractor  efforts  were  used  to  develop  system 
requirements.  Surveys  were  made  of  the  available  state-of-the-art  computer 
technology  to  ensure  that  the  requirements  established  were  practical  and 
feasible  within  available  resources.  Technical  data  requirements  were  based 
upon  experience  in  earlier  efforts  in  the  CMAS  program.  The  ABDA  requirements 
were  based  upon  in-house  efforts  and  a  contractual  effort  by  McDonnell -Douglas 
(Wilper,  Eschenbrenner,  &  Payne,  1983).  Requirements  for  the  diagnostics 
capabilities  were  based  upon  ongoing  in-house  work  to  develop  improved 
automated  diagnostics  techniques.  An  "A  Specification"  detailing  the 
technical  requirements  for  the  portable  unit  was  developed  under  contract  by 
Systems  Exploration,  Inc.  (Systems  Exploration,  Inc.,  1984).  This  document 
was  used  to  establish  contract  technical  requirements  for  the  PCMAS  hardware 
and  software. 


System  Development 

A  contract  was  awarded  on  31  July  1985,  to  Boeing  Military  Aircraft 
Company,  Huntsville,  Alabama,  for  the  development  of  the  PCMAS.  The  contract 
called  for  the  development  of: 

1.  The  PCMAS  computer  hardware  (three  units,  with  an  option  for  an 
additional  five  units). 

2.  System  software  for  the  PCMAS  (a  UNIX-based  operating  system  with 
utilities). 

3.  Applications  software  (for  presentation  of  technical  data;  automated 
diagnostics;  and  specialized  ABDA  data,  including  peel-away  graphics). 

4.  Sample  routine  maintenance  technical  data  for  a  subsystem  of  the  F- 16. 

5.  Sample  ABDA  technical  data. 


Project  Status 

Work  on  the  Boeing  contract  has  been  completed.  The  PCMAS  hardware  and 
software  have  been  delivered.  However,  contract  funds  ran  out  before  the 
hardware  could  be  fully  tested  and  before  all  known  "bugs"  in  the  software 
could  be  resolved.  Work  is  presently  in  progress  to  evaluate  the  PCMAS 
hardware  to  determine  if  additional  modifications  are  required.  In  addition, 
the  software  is  presently  being  evaluated  to  Identify  problem  areas. 
Necessary  modifications  to  the  hardware  and  software  will  be  made  in-house 
with  some  contractor  support.  Also,  when  the  problems  have  been  corrected, 
additional  units  will  be  constructed  for  use  in  planned  field  tests.  The  new 
units  will  Incorporate  an  Improved  graphics  board  and  a  refined  central 
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processor  board,  which  have  become  available  since  the  original  units  were 
constructed. 


System  Description 

The  complete  PCMAS  consists  of  a  small,  portable,  ruggedized, 
general-purpose  computer;  peripheral  units  (which  with  the  computer  comprise  a 
workstation);  system  software;  and  applications  software.  The  system  is 
designed  to  operate  in  two  modes:  with  the  portable  computer  in  a  stand-alone 
mode  (for  use  on  the  flightline),  and  with  microcomputers  in  a  workstation 
mode  (for  use  in  the  shop  and  for  software  development).  The  system 
specifications  are  presented  in  Table  11.  The  components  of  the  system  are 
described  below. 


Table  11.  PCMAS  Specifications 


Feature 

SDecification 

Dimensions 

15  x  12  x  3  inches 

Weight 

13  pounds 

Power  consumption 

46  watts  without  1553  bus 

52  watts  with  1553  bus 

Memory 

Internal 

CMOS  RAM 

EPROM 

Removable  cartridge 

2.0  megabytes 

1.5  megabytes 

1.0  megabyte  OIOS  RAM 

Processor 

Manufacturer/model 

Clock  speed 

Motorola  68010 

10MHz 

Display 

Type 

Active  display  area 
Resolution 

Electrol umi nescent 

7.68  x  3.83  inches 

512  x  256  pixels 
(66.6  pixels  per  inch) 

Keyboard 

Stand-alone  mode 

Special  purpose 

10  numeric  keys 

8  function  keys 

4  cursor  keys 

Enter  key 

Backspace  key 

Select  key 

Workstation  mode 


Modified  VT-100 


Table  11  (Concluded) 


Feature 

Specification 

Voice  recognition 

Type 

Speaker  dependent 

Capacity  i 

Recognizes  100  words 

Accuracy 

95%  recognition  in  85  db  noise 

Recognition  speed 

Less  than  .5  second 

Microphone 

Hands-free  headset 

Power  sources 

Battery  pack 

3  hours 

28  VDC 

Aircraft  power  (28  VDC) 

12  VDC 

Adapter  (28  VDC  output) 

110  VAC 

Adapter  (28  VDC  output) 

Interfaces 

Stand-alone  mode 

1553  bus 

MIL-STD-1553B 

Workstation  mode 

Standard 

RS-232 

Standard 

Centrex 

Ruggedization 

Shock 

Withstand  3-foot  drop 

Temperature  range 

32  to  100  degrees  F 

Moisture 

Operate  in  20-95%  humidity 
Operate  in  rain  w /  20  mph  wind 

Atmospheric  pressure 

Operate  at  6,000  feet 

Oil/chemical/dirt 

Resist  dirt,  oil,  fluid  spills 
Operate  in  chemical  environment 

Hardware 


Portable  Computer.  The  portable  computer  (Figure  51)  was  designed  and 
built  specifically  for  this  effort.  Off-the-shelf  components  were  used  where 
possible.  However,  some  major  modifications  and  redesign  of  some  components 
were  necessary.  The  system  is  designed  to  be  convenient  to  use  on  the 
flightline.  It  is  lightweight,  compact  enough  for  easy  handling,  and  easy  to 
operate.  It  is  sufficiently  rugged  to  withstand  the  rigors  of  testing  under  a 
variety  of  operational  conditions  on  the  flightline.  However,  It  Is  not 
designed  to  meet  full  military  specifications  for  ruggedization. 

The  PCMAS  portable  unit  display  is  an  electroluminescent  (EL)  panel.  The 
EL  panel  is  a  light-emitting  display.  It  provides  a  wide  angle  of  view  (30 
degrees)  and  is  suitable  for  use  under  most  lighting  conditions.  However, 
problems  are  encountered  In  direct  sunlight.  To  overcome  this  problem,  the 
display  is  laminated  with  a  polarizing  material,  and  a  shade  is  provided.  The 
EL  panel  has  a  moderately  high  resolution  (66.6  pixels  per  inch)  and  Is 
suitable  for  presenting  graphics  of  the  type  required  for  technical  data.  The 
EL  panel  used  for  the  PCMAS  portable  unit  Is  very  similar  to  the  panel  used  In 
the  Grid  Compass  Model  1139  computer  used  for  CMAS  II. 


Figure  51.  Portable  Computer-Based  Maintenance  Aids  System. 


A  special-purpose  keypad  was  developed  for  the  PCMAS  portable  unit.  The 
keypad  provides  a  standard  numeric  keypad  for  entry  of  numeric  data, 
programmable  function  keys  for  controlling  data  recall  and  presentation, 
cursor  control  keys,  an  enter  key,  a  backspace  key,  and  a  select  key  for  data 
input.  A  capability  to  input  alphabetic  characters  is  provided  using  a  menu 
select  technique  (t^e  cursor  and  select  keys  are  used  to  select  alphabetic 
characters  displayeu  on  the  screen).  The  keypad  is  sealed  to  provide 
protection  from  fluid  spills. 

Technical  data  for  presentation  on  the  portable  computer  are  stored  in 
interchangeable  auxiliary  memory  modules.  In  addition  to  storing  technical 
data,  the  memory  modules  can  be  used  to  record  Information  developed  during 
the  maintenance  task  (such  as  task  performed,  time  to  complete  the  task). 
This  task  can  be  downloaded  to  the  workstation  or  another  data  base  when  the 
task  is  completed.  In  addition,  the  memory  module  may  be  used  to  store 
applications  programs  that  are  not  maintained  in  the  built-in  EPROM 
(electronically  programmable  read  only  memory).  The  initial  auxiliary  memory 
modules  have  a  capacity  of  1  megabyte.  It  is  anticipated  that  advances  in 
memory  technology  will  make  a  6-megabyte  module  possible  in  the  near  future. 

A  built-in  MIL-STD-1 5536  bus  interface  provides  a  capability  to 
:ommunicate  directly  with  on-aircraft  systems  for  aircraft  equipped  with  a 
MIL-STD-1 5538  data  bus.  The  1553B  interface  is  capable  of  performing  as  a  bus 
controller,  remote  terminal,  or  as  a  bus  monitor  on  a  1553B  data  bus.  This 
feature  will  make  it  possible  for  the  PCMAS  portable  unit  to  Interrogate  the 
aircraft's  built-in  tests  (BIT)  to  obtain  performance  data.  The  PCMAS 
portable  unit  then  can  use  this  information,  with  its  Internal  diagnostic 
routines,  to  diagnose  system  faults. 


The  PCMAS  portable  unit  operates  on  28  VDC  from  any  of  four  power 
sources:  an  external  battery,  aircraft  power,  a  12- VDC  adapter,  and  a  110- VAC 


adapter.  The  PCMAS  battery  uses  18  1.49-VDC  Silver-Zinc  cells  assembled  in  a 
battery  pack.  The  battery  pack  is  external  to  the  PCMAS  unit.  A  6-foot  cable 
is  provided  with  the  battery  pack,  which  connects  directly  into  the  unit.  The 
battery  pack  is  rechargeable  and  has  a  battery  life  between  50  and  100 
cycles.  The  PCMAS  computer  will  operate  for  3  hours  from  the  battery  pack. 

Workstation.  The  PCMAS  workstation  consists  of  the  portable  PCMAS  unit,  a 
junction  box,  a  modem,  a  Winchester  disk  drive,  a  keyboard,  and  printer.  The 
workstation  can  be  interfaced  to  a  Sun  Microsystems  workstation  for  use  in 
data  development,  in  software  development,  or  to  simulate  communications  with 
other  computer  systems.  The  workstation  mode  is  provided  for: 

1.  Developing  and/or  modifying  software. 

2.  Loading  data  onto  auxiliary  modules  from  the  Winchester  disk  drive. 

3.  Interfacing  with  other  computer  systems. 

4.  Downloading  data  from  the  auxiliary  memory  module  to  the  Winchester 
disk  drive  or  to  other  computer  systems. 

5.  Printing  data  or  software  from  the  auxiliary  memory  module,  from  the 
Winchester  disk,  or  from  another  computer  system. 

The  PCMAS  portable  unit  connects  to  the  workstation  via  a  specially 
developed  junction  box.  The  junction  box  contains  connections  to  the  keyboard, 
the  Winchester  disk  drive,  a  Sun  workstation,  the  printer,  an  auxiliary 
display,  and  the  modem.  In  addition  to  the  PCMAS  portable  unit,  the  following 
equipment  is  Included  as  part  of  the  PCMAS  workstation: 

Junction  Box:  Developed  by  Boeing  specifically  for  this  application. 

Disk  Drive:  Javelin  Emulex,  Model  ED2/40-T  Winchester  disk  drive,  with  a 
streaming  tape  cartridge  tape  drive. 

Keyboard:  Digitran,  Model  220-8. 

Printer:  Okidata  Microline,  Model  192  Personal  Printer. 

Modem:  Datec,  Model  PAL  212  (300/1200). 


Software 


Two  basic  types  of  software  were  developed  for  the  PCMAS:  operating 
system  software  and  applications  software.  Commercially  available  software 
was  used  where  possible  and  cost  effective.  In  those  cases  where  new  software 
development  was  required,  the  software  was  developed  on  a  Sun  workstation  and 
transported  to  the  PCMAS.  (A  system  requirement  was  that  software  developed 
on  the  Sun  workstation  run  on  the  PCMAS  without  modification.)  The  software 
available  for  the  PCMAS  Is  described  briefly  in  the  following  paragraphs. 

System  Software.  The  PCMAS  operating  system  Is  a  kernel  of  the  Berkley 
UNIX  4.2  operating  system.  The  operating  system  contains  only  those  functions 
of  the  UNIX  operating  system  required  to  support  the  PCMAS. 
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Applications  Software.  Applications  software  was  developed  or  is  under 
development  for  the  presentation  of  technical  data,  for  interaction  with  the 
aircraft  15538  data  bus,  and  for  presentation  of  diagnostic  procedures.  It  is 
anticipated  that  additional  applications  software  will  be  developed  for 
studies  in  areas  such  as  the  presentation  of  training  and  maintenance  data 
collection  in  support  of  the  IMIS  program.  The  following  paragraphs  provide  a 
brief  description  of  software  packages  developed  or  under  development  for  use 
with  the  PCMAS: 

1.  Boeing  Presentation  Software.  As  part  of  the  PCMAS  contract,  Boeing 
developed  software  for  the  presentation  of  technical  data.  The  software 
provides  essentially  the  same  capabilities  as  the  CMAS  II  software  with  the 
addition  of  a  rescale  (or  zoom)  capability.  In  addition  to  the  presentation 
software,  limited  software  was  developed  for  the  creation  of  data  for 
presentation  on  the  PCMAS.  The  capabilities  of  this  software  are  limited.  As 
a  result,  the  creation  of  data  for  presentation  using  this  software  is  a 
labor-intensive  task. 

2.  Boeing  Maintenance  Diagnostics  Information  System  (MDIS)  Software. 
The  Boeing  Company  had  previously  developed  software  for  an  automated 
diagnostic  system.  The  MOIS  software  was  transported  to  the  PCMAS  to  provide 
an  automated  diagnostics  capability.  However,  the  MDIS  software  Is 
proprietary.  This  limits  its  usability  since  modifications  required  to 
operate  the  software  with  the  1553B  bus  are  not  possible  without  additional 
contractual  arrangements  with  Boeing. 

3.  AFHRL  Authoring  and  Presentation  System  (APS)  Software.  AFHRL  is 
developing  a  system  for  authoring  and  presenting  technical  data  for  use  with 
an  automated  system.  The  system  is  designed  to  use  a  "neutral"  data  base 
which  is  suitable  for  presentation  on  a  variety  of  computer  systems  without 
modification  of  the  data.  The  presentation  software  will  be  adapted  for  use 
with  the  PCMAS.  This  will  make  it  possible  to  use  the  PCMAS  to  present 
technical  data  from  the  neutral  data  base. 

4.  AFHRL  Diagnostics  Software.  In  a  combination  In-house/contract 
effort,  AFHRL  has  developed  software  for  an  advanced  approach  to  automated 
diagnostics.  This  software  will  be  transported  to  the  PCMAS  and  expanded  to 
Incorporate  the  capability  to  Interact  with  the  1553B  bus.  A 


Planned  Efforts 

PCMAS  Technical  Data  Evaluation 

An  evaluation  of  the  PCMAS  for  delivering  technical  data  for  routine 
maintenance  is  scheduled  for  the  Spring  of  1988.  For  this  effort,  technical 
data  for  the  Chaff /Flare  Distribution  System  of  the  F-16  will  be  converted  to 
an  automated  data  base  using  the  AFHRL  Authoring  and  Presentation  System. 
These  data  will  then  be  used  with  the  PCMAS  to  evaluate  the  PCMAS  as  a  device 
for  presenting  technical  data  for  use  on  the  flightline.  Items  of  concern 
will  include: 


1.  The  degree  to  which  the  system  is  accepted  by  the  user 


2.  Identification  of  any  problems  encountered  In  using  the  system  on  the 
flightline  (problems  in  reading  the  display,  finding  a  suitable  place  to 
locate  the  unit  while  working,  etc.). 

3.  The  effectiveness  of  the  PCMAS  for  presenting  technical  Information. 

4.  Identification  of  potential  improvements  of  the  system. 


ABDA  Evaluation 


The  PCMAS  will  be  used  with  a  special  ABDA  data  base  to  evaluate  the 
concept  of  an  automated  ABDA.  An  ABDA  data  base  will  be  developed  for  a 
section  of  an  F-4  aircraft.  The  PCMAS  and  the  data  base  will  then  be 
evaluated  as  an  automated  ABDA  by  having  battle  damage  assessors  use  the  PCMAS 
to  assess  a  damaged  F-4  aircraft.  The  study  will  examine: 

1.  The  effectiveness  of  the  PCMAS  for  presenting  the  required  data. 

2.  The  adequacy  of  the  ABDA  data  base. 

3.  Problems  Involved  In  using  the  PCMAS  In  the  battle  damage  assessment 
environment. 

4.  The  acceptance  of  the  system  by  the  battle  damage  assessors. 

5.  Identification  of  potential  Improvements  In  the  PCMAS. 


Diagnostic  Evaluations 

Two  studies  will  be  accomplished  to  evaluate  the  PCMAS  and  AFHRL 
diagnostic  techniques.  In  the  first  study,  software  will  be  developed  to 
adapt  the  AFHRL  diagnostic  algorithms  to  use  data  from  the  F-16  data  bus. 
Diagnostic  and  technical  data  bases  will  be  developed  for  two  or  more 
subsystems  of  the  F-16  which  are  serviced  by  the  1553B  data  bus.  The  data 
bases  will  then  be  used  to  evaluate  the  PCMAS  and  diagnostic  algorithms  as 
diagnostic  aids.  This  will  be  accomplished  by  having  technicians  use  the 
system  to  diagnose  known  faults  In  the  testbed  subsystems. 

In  the  second  study,  a  similar  evaluation  will  be  accomplished  In  a  joint 
effort  with  the  Navy  using  the  A/F-18  aircraft  as  a  testbed.  The  A/F-18  Is  a 
newer  aircraft,  with  more  sophisticated  electronics  and  Improved  self-test 
capabilities.  Use  of  the  A/F-18  will  allow  the  evaluation  of  diagnostic 
capabilities  not  available  on  the  F-16.  Software  will  be  developed  to  adapt 
the  diagnostic  algorithms  to  use  data  from  the  A/F-18  data 'bus.  Technical 
data  bases  will  be  developed  for  two  or  more  subsystems  of  the  A/F-18  which 
are  serviced  by  the  1553  data  bus.  The  data  bases  will  then  be  used  to 
evaluate  the  PCMAS  and  algorithms  as  diagnostic  aids.  This  will  be 
accomplished  by  having  technicians  use  them  to  Identify  faults  in  the 
aircraft.  The  evaluations  will  examine: 

1.  The  effectiveness  of  the  PCMAS  and  diagnostic  routines  as  diagnostic 
aids. 
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2.  The  problems  Involved  In  using  the  PCMAS  as  a  diagnostic  aid. 

3.  The  acceptance  of  the  PCMAS  by  the  technicians. 

4.  Potential  Improvements  of  the  PCMAS  as  a  diagnostic  aid. 


VII.  DISCUSSION 

This  project  has  clearly  demonstrated  that  a  computer-based  maintenance 
aids  system  Is  feasible.  It  has  shown  that  such  a  system  Is  well  received  by 
the  technicians  who  will  use  It  and  that  technicians  can  effectively  use  an 
automated  system  to  perform  maintenance.  In  addition.  It  has  provided 
evidence  that  technicians  are  able  to  perform  Intermediate  level 
troubleshooting  on  electronic  equipment  more  effectively  when  using  an 
automated  technical  data  presentation  system  than  when  using  conventional  TOs. 

The  CMAS  II  system  was  shown  to  be  an  effective  system.  However,  It  must 
be  considered  a  first-generation  automated  technical  data  presentation 
system.  Although  some  preliminary  prototyping  was  done,  the  design  CMAS  II  Is 
largely  based  on  logical  analyses  of  the  requirements  for,  the  potential 
approaches  for,  and  problems  of  presenting  technical  data  on  an  automated 
system.  The  time  and  resources  were  not  available  to  conduct  detailed  design 
studies  to  resolve  each  design  Issue.  As  a  result,  most  of  the  features  of 
the  CMAS  II  are  based  upon  the  best  judgments  of  researchers  who  are  familiar 
with  state-of-the-art  techniques  for  presenting  technical  data,  MMI 
techniques,  human  factors  technology,  and  computer  science  ("given  what  we 
know  about  the  problem  and  similar  applications,  we  think  that  this  Is  the 
best  way  to  do  It").  In  some  cases,  system  features  and  capabilities  were 
determined  by  the  limitations  of  the  computer  technology  available  at  the 
time.  Fortunately,  most  of  the  judgments  made  In  designing  the  CMAS  II  were 
correct,  and  the  system  concept  was  proven  to  be  successful. 

It  Is  recognized  that  the  CMAS  II  Is  not  the  ultimate  system.  It  Is  a 
system  that  has  been  shown  to  work  and  Is  as  advanced  as  any  system  known  to 
be  In  existence  today.  Although  It  Is  not  the  "ultimate"  system.  It  should  be 
considered  a  starting  point  for  the  development  of  more  advanced  systems.  Its 
features  should  definitely  be  considered  In  the  development  of  any  system 
planned  for  operational  use  In  the  future. 


Observat 1 ons/Conc 1 us 1 ons 

The  development  of  the  CMAS  II  and  Its  predecessors  has  provided  a  great 
deal  of  valuable  Information  relative  to  the  development  of  automated 
technical  data  presentation  systems.  It  provides  the  basis  for  further 
refinement  of  the  technology  and  for  the  development  of  systems  for 
operational  use.  Some  key  observations  and  conclusions  from  the  project  are 
discussed  below. 

User  Acceptance.  Maintenance  technicians  will  accept  an  automated 
technical  data  presentation  system,  provided  it  Is  easy  to  use  and  effectively 
meets  their  needs  for  Information.  It  Is  essential  that  the  system  meet  the 
technicians'  needs  at  least  as  well  as  the  paper-based  system.  The  experience 

i 

i 
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with  the  CMAS  I  at  Offutt  AFB  clearly  demonstrated  how  rapidly  the  technicians 
will  reject  an  automated  system  that  does  not  meet  their  needs. 

There  are  several  factors  which  affect  user  acceptance.  Response  time  is 
a  critical  factor.  The  technician  will  not  wait  very  long  for  the  system  to 
present  Information.  Acceptable  times  between  the  request  for  and  display  of 
Information  are  not  well  defined.  The  "acceptable"  time  appears  to  vary  with 
the  type  of  data  being  requested.  For  recall  of  the  next  frame  in  a  sequence 
of  data  (such  as  a  sequence  of  procedural  frames) ,  only  short  times  (less  than 
5  seconds)  would  be  tolerated.  Technicians  seem  to  be  willing  to  tolerate 
longer  delays  if  they  realize  that  Information  requested  Is  for  a  different 
type  of  data,  must  be  retrieved  from  a  different  part  of  the  data  base,  or  is 
a  large  file  (such  as  a  schematic)  which  requires  longer  to  load.  The  field 
tests  conducted  In  the  project  provided  only  tentative  information  on 
acceptable  time  delays  since  the  technicians  had  the  opportunity  for  only 
limited  use  of  the  system.  Their  reactions  may  be  quite  different  with 
continued  exposure  In  an  operational  setting.  It  is  clear,  however,  that  a 
rapid  response  time  Is  critical  to  the  acceptance  of  the  system.  It  Is  likely 
that  technicians  will  become  much  less  tolerant  of  long  delays  when  working 
under  the  pressures  of  the  operational  environment. 

Ease  of  use  Is  essential  for  user  acceptance.  The  actions  necessary  to 
retrieve  a  given  piece  of  Information  should  be  logical  and  somewhat  obvious. 
They  should  not  be  cumbersome  or  require  unnecessary  steps  (e.g.,  the  CMAS  I 
requirement  to  respond  "yes"  or  "no"  to  the  use  of  the  graphics  mode  before 
going  to  the  next  frame).  They  should  require  a  minimum  of  keystrokes  for  any 
action.  Its  simplicity  of  use  was  a  major  reason  for  the  success  of  CMAS  II. 

The  MM  I  techniques  used  In  the  CMAS  II  were  effective  and  made  Important 
contributions  to  the  acceptance  of  the  system.  They  made  the  system  easy  to 
learn  and  use.  The  technicians  experienced  very  little  difficulty  In  using 
the  system  after  a  few  minutes  of  training  and  practice.  The  MMI  techniques 
provided  the  flexibility  to  move  rapidly  In  the  data  base  and  retrieve  all 
required  Information.  This  flexibility  eliminated  the  "trapped"  feeling 
reported  by  technicians  using  the  CMAS  I. 

Accurate  technical  data  are  essential  to  user  acceptance  for  automated 
technical  data  presentation  systems.  Errors  In  the  technical  data  were 
critical  factors  In  the  rejection  of  the  CMAS  I  by  the  technicians.  Technical 
accuracy  Is  essential  for  all  technical  data,  but  It  Is  an  especially 
difficult  problem  for  automated  technical  data  presentation  systems  since 
there  are  many  more  opportunities  for  error.  This  is  due  to  the  requirement 
to  Include  codes  to  control  branching  to  the  next  frame  or  optional  frames  of 
data.  Branching  errors  can  be  critical  If  they  cause  the  technician  to  get 
lost  in  the  data  base  or,  even  worse,  cause  him  to  skip  a  critical  step. 
Quality  control  for  branching  sequences  is  very  difficult  to  adequately 
accomplish.  Given  the  flexibility  built  into  an  automated  system.  It  Is 
practically  Impossible  to  check  every  possible  branching  sequence  for 
accuracy.  The  use  of  automated  authoring  aids  to  assist  in  inserting  and 
verifying  the  accuracy  of  branching  instructions  is  the  most  promising  (though 
untried)  approach  to  resolving  this  problem. 

The  acceptance  of  an  automated  system  by  the  technicians  Is  facilitated  by 
the  relatively  high  computer  literacy  of  Air  Force  maintenance  personnel. 


These  personnel  are  used  to  working  with  high  technology  equipment  and  are  not 
awed  by  the  computer  system,  as  might  be  the  case  with  a  less  sophisticated 
group.  Many  technicians  have  home  computers  and  are  very  knowledgeable  of 
computer  systems.  This  computer  literacy  could.  In  fact,  be  a  potential 
problem  since  future  "hackers"  may  find  the  temptation  to  tamper  with  the 
system  Irresistible.  Appropriate  safeguards  will  be  required. 

System  Efficiency.  The  project  has  demonstrated  that  an  automated 
technical!  data  presentation  system  can  effectively  present  technical  data  for 
use  by  maintenance  technicians.  It  has  provided  evidence  that  such  a  system 
can  provide  Information  at  least  as  effectively  as  a  paper-based  system  and, 
for  at  least  some  applications,  more  effectively.  However,  much  more  data  are 
needed  to  permit  firm  conclusions  on  this  Issue.  The  AFHRL  and  NPRDC  field 
tests  of  the  CMAS  provided  valuable  Information  on  the  Issue.  However,  the 
tests  were  not  adequate  to  provide  firm  answers.  The  tests  used  very  small 
samples  of  tasks  and  were  limited  to  an  electronic  system.  In  addition,  the 
troubleshooting  procedures  were  designed  to  Isolate  known  faults.  To  provide 
comprehensive  and  final  answers,  study  Is  needed  which: 

1.  Uses  a  larger  sample  of  tasks  (remove  and  replace,  alignment, 
checkout,  troubleshoot,  etc.). 

2.  Uses  data  from  different  types  of  systems  (electronic  and  mechanical 
systems). 

3.  Uses  data  for  a  complete  subsystem.  These  data  should  be  developed 
using  the  procedures  that  will  be  used  for  development  of  automated  data  for 
operational  application.  Troubleshooting  data  should  cover  all  Identifiable 
failures. 

4.  Compares  performance  using  the  automated  system  with  performance  using 
paper-based  conventional  technical  manuals,  Job  Guide  manuals,  and  fault 
Isolation  manuals.  (Comparison  of  performance  with  Job  Guide  manuals  and 
fault  Isolation  manuals  Is  necessary  to  ensure  that  gains  In  performance  are 
not  simply  due  to  reformatting  rather  than  to  presentation  on  the  automated 
system. ) 


Recommendations  for  a  Next-Generation  System 

As  indicated  above,  the  CMAS  II  cannot  be  considered  the  ultimate  system 
for  the  storage,  retrieval,  and  presentation  of  technical  data  on  a  computer 
display.  As  the  technology  develops,  significant  Improvements  will  be  made. 
However,  the  CMAS  II  and  the  other  research  efforts  conducted  under  this 
project  do  provide  a  sound  basis  for  recommendations  for  the  development  of 
the  next-generation  automated  technical  data  presentation  system. 
Recommendations  and  guidelines  for  the  development  of  future  automated 
technical  data  presentation  systems  are  provided  in  the  following  paragraphs. 


Response  Times 

The  optimal  time  to  retrieve  and  display  the  "next"  frame  of  procedural 
data  In  a  sequence  should  be  1  second  or  less,  and  should  not  exceed  2 
seconds.  The  time  to  display  a  large  graphic  should  not  exceed  5  seconds. 
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The  tine  to  retrieve  data  for  a  new  procedure  or  fron  another  part  of  the  data 
base  should  not  exceed  5  seconds.  As  a  general  rule,  response  tines  should  be 
kept  to  the  nlnlnun  that  the  hardware  and  software  can  support.  An 
"Instantaneous"  response  Is  not  too  fast. 


Data  Base  Manipulation 


The  system  should  provide  naxlnun  flexibility  to  permit  easy  retrieval  and 
manipulation  of  data  fron  the  data  base.  As  a  general  guideline,  the  minimum 
number  of  keystrokes  should  be  required  to  achieve  any  required  action.  In 
most  cases,  no  more  than  one  keystroke  should  be  required  to  retrieve  the  next 
piece  of  Information  (l.e.,  next  frame  of  data,  next  menu,  etc.). 

Data  Access  Methods.  The  following  methods  should  be  provided  for 
locating  and  accessing  Information  In  the  data  base: 

1.  Menu  Access.  The  user  should  be  able  to  access  any  desired 
Information  from  a  series  of  top-down  menus  (system  general  to  system 
specific).  The  number  of  menus  required  to  Identify  the  Information  should 
not  exceed  four  In  most  cases.  The  number  of  Items  per  menu  should  not  exceed 
10  to  permit  selection  of  an  Item  with  the  press  of  one  numeric  key.  An 
alternate  approach  for  selecting  from  the  menu  Is  to  use  the  cursor  and  select 
keys  to  make  a  selection.  However,  this  process  requires  more  actions  by  the 
user  and  Is  much  slower  unless  a  mouse  or  other  device  which  provides  for 
quick  cursor  movement  Is  used. 

The  number  of  menus  required  to  Identify  a  specific  piece  of  data  Is,  In 
part,  a  function  of  the  level  that  the  menu  system  Is  entered  (e.g.,  starting 
with  a  menu  for  the  overall  aircraft  versus  starting  with  a  menu  for  a 
specific  subsystem).  As  a  means  of  limiting  the  number  of  menus  which  must  be 
traversed.  It  Is  recommended  that  the  capability  be  provided  for  the 
supervisor  or  system  manager  to  specify  the  level  of  the  first  menu  presented 
when  the  user  signs  on.  For  example,  the  supervisor  of  a  radar  shop  could 
preset  the  computers  In  the  shop  to  first  present  a  menu  for  the  radar  systems 
maintained  In  that  shop.  An  alternate  approach  would  be  to  allow  the  users  to 
specify  the  menu  to  appear  as  the  first  frame  when  they  sign  on  to  the  system. 

2.  Direct  Access.  The  system  should  provide  the  capability  for  the  user 
to  go  directly  to  a  given  procedure  or  Item  of  data  by  Inputting  a  procedure 
title,  section  title,  MIDAS  code,  part  number,  reference  designator,  frame 
number,  or  other  unique  Identifier.  In  those  cases  where  there  may  be  more 
than  one  piece  of  Information  available  relevant  to  an  Identifier,  the  system 
should  respond  with  a  menu  Identifying  the  available  Information  for  that 
Identifier.  The  user  should  then  be  able  to  recall  the  deslrqd  Information  by 
selecting  from  the  menu. 

3.  User-Defined  Menu.  The  users  should  be  able  to  "construct*  their  own 
menus  tailored  to  their  own  needs.  This  will  allow  them  to  list  only  those 
Items  which  are  frequently  used.  The  system  should  provide  the  capability  to 
keep  Identifying  this  menu  as  their  "main  menu,"  which  automatically  appears 
when  they  sign  on  or  when  they  press  the  menu  or  Table  of  Contents  function 
key.  Implementation  of  this  capability  should  provide  for  the  user-defined 
menu  to  always  Include  the  option  to  go  to  the  system  main  menu  or  to  the 
direct  access  mode. 
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4.  Assignment  Index  Access.  A  capability  should  be  provided  for  the 

supervisor  to  use  the  system  to  wake  work  assignments.  When  the  technicians 
sign  on  and  select  the  assignment  roster,  they  should  be  able  to  use  the 

roster  as  a  menu  to  select  the  data  for  their  assigned  task.  (Note:  A 
similar  capability  was  Included  in  the  CMAS  I  system.  However,  it  was  not 
tested  since  the  system  was  not  used  for  actual  maintenance  tasks.) 

5.  I PB  Access.  The  location  of  IPS  Information  in  the  data  base  presents 

some  special  problems.  The  user  may  know  the  item  nomenclature,  part  number, 
or  reference  designator.  In  this  case,  the  desired  information  can  be 

recalled  using  the  direct  access  mode.  If  the  system  name,  subsystem  name, 
and  assembly  name  are  known  but  the  name  or  number  of  the  specific  part  is 
not,  the  menu  system  can  be  used  to  locate  a  drawing  of  the  assembly. 
However,  the  user  can  go  no  further  unless  the  Item  of  Interest  can  be 

Identified  and  the  specific  Information  can  be  recalled.  A  "graphic"  special 
menu  is  recommended  for  this  purpose.  An  exploded  view  of  the  assembly  can  be 
used  as  the  graphic  menu.  Callouts  p r  other  means  can  be  used  to  identify 
specific  parts.  By  entering  the  callout  number  or  other  Identifier  for  the 
Item  of  Interest,  the  user  can  recall  a  detailed  drawing  of  and  parts 
information  on  the  Item. 

Capabilities  for  Movement  within  the  Data  Base.  Once  the  user  has  located 
the  desired  information,  the  system  should  provide  extensive  capabilities  for 
movement  within  the  data  base  to  obtain  additional  Information.  The  following 
capabilities  should  be  provided: 

1.  6o  to  next  step.  A  function  key  should  be  provided  to  go  to  the  next 
step  In  a  sequence .  this  function  should  be  active  except  when  the  next  step 
Is  based  upon  conditional  branching  or  when  the  user  Is  asked  to  select  from  a 
menu  of  options. 

2.  Back  up  In  a  procedure.  A  function  key  should  be  provided  to  permit 

the  user  to  step  backward  In  the  procedure.  Pressing  the  BACK  function  key 
should  take  the  user  to  the  preceding  step  In  the  procedure.  Repeated 

pressing  of  the  BACK  key  should  lead  the  user  to  the  first  step  of  the 
procedure. 

3.  6o  to  previous  step.  A  function  key  should  be  provided  to  return  to 

the  previous  frame  In  a  sequence.  Repeated  pressing  of  the  RETRACE  key  should 
permit  the  users  to  retrace  their  steps  through  the  data.  This  function 

should  always  be  active. 

4.  Conditional  branching.  The  system  should  have  the  capability  to 

branch  to  the  next  appropriate  frame  as  determined  by  the  user's  response 

("yes,"  "no,"  or  other  input)  to  a  question  presented  by  the  current  frame. 

5.  Select  an  option.  The  system  should  allow  the  user  to  select  from 

several  options  (Identified  by  a  footer  at  the  bottom  of  each  frame).  In  most 
cases,  the  option  should  be  selected  by  pressing  a  function  key.  However,  If 
there  Is  more  than  one  option  available  for  that  category  of  data,  a  pop-up 
menu  should  be  provided  listing  the  available  options  (e.g.»  If  the  users 
select  a  "diagrams"  option,  they  may  be  presented  with  a  menu  listing 

schematics,  functional  diagrams,  or  wiring  diagrams).  In  this  case,  the 
footer  should  Indicate  that  options  are  available  for  a  given  type  of  data 
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(such  as  diagrams).  The  press  of  an  options  function  key  should  retrieve  the 
options  nenu.  A  pop-up  menu  Is  suggested. 

6.  Return  after  branching.  A  function  key  should  be  provided  to  allow 
the  user  to  return  to  the  same  point  In  the  data  base  after  branching  to 
another  location  In  the  data  base.  For  example,  If  the  user  branched  to  a 
schematic,  pressing  the  RETURN  function  should  take  him  back  to  the  frame  from 
which  he  selected  the  schematic.  If  he  branched  from  the  schematic  to  the 
IPB,  pressing  RETURN  should  return  him  to  the  schematic.  Pressing  RETURN  a 
second  time  should  return  him  to  the  original  frame. 

7.  Access  other  data.  A  function  key  should  be  provided  to  allow  the 
user  to  return  to  the  main  (or  other)  menu  or  to  select  the  direct  access  mode. 

8.  6o  to  next  procedure.  At  the  end  of  a  procedure,  the  system  should 
provide  the  opportunity  for  the  user  to  go  directly  to  the  procedure  for  any 
required  follow-on  maintenance.  Similarly,  when  a  fault  has  been  Isolated  In 
a  troubleshooting  procedure,  the  system  should  provide  the  user  with  an  option 
to  go  directly  to  the  procedure  to  correct  the  fault. 

9.  Change  level  of  detail.  For  procedural  data,  a  function  key  should 
be  provided  to  give  the  user  the  capability  to  select  a  more-detailed  or 
less-detailed  presentation.  The  system  should  provide  the  capability  for  the 
supervisor  to  limit,  as  appropriate,  the  level  of  access  available  to  a  given 
user.  This  capability  may  be  necessary  to  ensure  that  less  experienced 
technicians  do  not  use  data  Intended  for  use  by  experienced  technicians  only. 

10.  Book  mark.  A  capability  should  be  provided  to  allow  the  users  to 
establish  a  book  mark  at  a  given  location  In  the  data  so  that  they  may  return 
directly  to  that  point  at  a  later  time  (either  In  the  same  session  or  In  a 
future  session)  without  going  through  the  menu  mode  or  remembering  and 
Inputting  an  Identifier  In  the  direct  access  mode. 

11.  Hold  feature.  A  capability  should  be  provided  which  allows  the  users 
to  place  a  frame  of  Information  on  hold  while  they  examine  a  second  frame  and 
to  switch  easily  back  and  forth  between  the  frames. 

Display  Manipulation  Capabilities.  The  following  capabilities  to  change 
the  manner  In  which  selected  information  Is  displayed  on  the  screen  should  be 
provided: 

1*  Scroll.  The  system  should  provide  the  capability  for  the  user  to 
"move"  the  display  window  over  a  drawing  that  Is  larger  than  the  display.  An 
Icon  should  be  provided  to  advise  the  user  that  the  graphic  Is  larger  than  the 
display  and  that  the  scroll  function  Is  active. 

2.  Rescale.  The  capability  should  be  provided  to  allow  the  user  to 
rescale  (enlarge  or  reduce)  graphics  presented.  Rescaling  should  be 
accomplished  In  a  continuous  fashion  (l.e.,  the  Illustration  should  appear  to 
"grow"  or  "shrink"  before  the  user's  eyes,  as  opposed  to  the  screen  being 
blanked  out  and  the  Illustration  redrawn  at  a  larger  or  smaller  size).  An 
appropriate  scaling  factor  should  be  used  so  that  the  difference  In  each 
succeeding  iteration  Is  not  so  large  that  the  users  must  reorient  themselves, 
but  not  so  small  that  an  excessive  number  of  Iterations  are  required  to  reach 


the  desired  size.  A  limiting  factor  should  be  provided  so  that  the 
illustration  cannot  be  enlarged  to  a  point  that  it  is  useless  (e.g . •  enlarged 
until  only  a  blank  screen  is  seen)  or  made  so  small  that  it  is  useless  (e.g., 
reduced  to  a  single  dot).  An  icon  should  be  provided  to  inform  the  user  that 
the  rescale  feature  is  available. 

The  capability  to  enlarge  the  graphic  is  needed  in  some  instances  where 
the  user  requires  additional  detail  or  must  view  the  illustration  from  a 
distance.  This  feature  is  most  useful  for  examination  of  IPB  illustrations  or 
complex  diagrams  such  as  schematics.  The  requirement  for  the  use  of  this 

feature  should  be  limited  If  the  graphic  is  initially  presented  at  an 

appropriate  level  of  detail  and  size. 

Computational  capability.  The  system  should  provide  a  capability  to 

perform  calculations  on  data  Input  by  the  user  (at  the  system's  request).  The 
system  should  present  the  results  of  the  calculations  and,  at  the  user's 
request,  branch  to  the  next  appropriate  frame  based  upon  that  result.  This 
feature  eliminates  the  requirement  for  the  technician  to  perform  the 

calculations  by  hand  and  reduces  the  risk  of  error. 

Input  capabilities.  An  effective  automated  technical  data  presentation 
system  must  provide  a  convenient  method  for  the  user  to  Input  a  request  or 
Input  data  to  the  computer.  Specially  designed  function  keyboards  should  be 
provided  for  this  purpose.  Since  requirements  for  In-shop  and  on-aircraft 
maintenance  are  somewhat  different,  separate  function  keyboards  are 
recommended  for  the  two  applications.  It  is  anticipated  that  shop 
applications  will  be  used  for  housekeeping  functions  (recording  maintenance 
actions,  data  base  maintenance ,  etc.),  as  well  as  for  presentation  of  data. 
Thus,  the  shop  system  will  require  a  convenient  method  for  Inputting 
alphanumeric  characters.  Requirements  for  Input  of  alphanumeric  characters  to 
the  portable  system  are  limited  and  can  be  met  by  ether  means  (such  as 

selection  of  characters  presented  on  the  display),  iince  space  on  the 
keyboard  for  a  portable  system  is  limited.  It  Is  suggested  that  the  alphabetic 
keys  not  be  Included. 

In  many  cases,  the  same  technicians  perform  both  In-shop  and  on-equipment 
maintenance.  Thus,  it  Is  highly  desirable  that  the  keyboards  for  the  shop  and 
portable  systems  be  as  similar  as  possible  so  that  the  technician  may  switch 
"comfortably"  between  the  keyboards.  The  function  keys,  numeric  keys,  and 
control  keys  should  be  placed  In  arrangements  that  are  as  similar  (similar 

locations,  similar  order  of  appearance,  etc.)  as  possible  while  still  leaving 
room  for  the  alphabetic  keys  on  the  shop-level  keyboard.  An  option  would  be 
to  use  two  keyboards:  a  function  keypad  for  using  the  system  as  a  maintenance 
aid,  and  a  full  alphanumeric  keyboard  for  the  occasional  In-shop  task  which 
requires  the  input  of  a  large  amount  of  alphanumeric  data  or  for  housekeeping 

tasks.  When  not  in  use,  the  second  keyboard  would  be  kept  In  a  convenient 

location  out  of  the  technician's  way  on  the  workbench. 

In  designing  function  keypads  for  In-shop  maintenance  and  for  on-equipment 
maintenance,  the  following  requirements  should  be  considered: 

1.  The  space  available  on  the  shop  workbench  is  limited.  The  keypad  must 
be  small  enough  to  allow  adequate  work  space.  The  space  available  on  the 
portable  unit  is  also  very  limited  (and  will  become  even  more  limited  as 
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displays  become  larger).  Thus,  it  is  essential  that  the  keypad  be  kept  as 
small  as  possible. 


2.  The  keyboard  must  be  rugged  and  spill  proof. 

3.  The  keyboard  for  the  shop  should  be  connected  to  the  computer  system 
by  a  cable,  so  that  it  can  be  moved  to  the  most  convenient  location  on  the 
workbench  without  moving  the  computer  or  display. 

4.  The  keyboard  for  the  portable  unit  must  be  suitable  for  use  when 
wearing  gloves  provided  with  the  chemical  defense  ensemble. 

The  most  important  components  of  the  function  keyboard  are  the  function 
keys.  It  is  recommended  that  a  combination  of  programmable  and  "hard-wired" 
function  keys  be  used.  Hard-wired  function  keys  should  be  used  for  functions 
that  are  always  or  almost  always  active.  These  Include  functions  such  as 
next,  back,  and  data  access.  The  remaining  keys  should  be  used  for  functions 
that  are  not  always  active,  such  as  for  schematic  drawings  and  parts 
Information.  The  allocation  of  function  keys  must  be  based  upon  an  analysis 
of  the  requirements  of  the  specific  application.  It  is  recommended  that  the 
programmable  function  keys  for  the  portable  system  be  placed  In  a  row  below 
the  screen  so  that  the  function  of  the  keys  can  be  defined  in  a  footer  at  the 
bottom  of  the  display  (1:1  correspondence).  Hard-wired  function  keys  need  not 
be  located  at  the  bottom  of  the  screen.  However,  they  must  be  located  in  a 
convenient  position  and  should  be  in  a  logical  arrangement  (l.e.,  grouped 
functionally,  spatially,  etc.).  When  an  inactive  function  key  is  pressed, 
some  type  of  feedback  (such  as  an  audible  "beep")  should  be  provided 
Indicating  that  the  function  Is  not  active.  Also,  a  provision  should  be  made 
for  quick  recovery  If  the  user  presses  the  wrong  function  key.  A  similar 
keyboard  arrangement  should  be  used  for  the  shop  system  for  consistency. 

Although  not  evaluated  In  this  project,  two  other  data  input  devices 
should  be  considered.  They  are  voice  activation  systems  and  touch  panel 
systems.  Voice  activation  systems  have  the  potential  to  be  an  effective  tool 
for  automated  technical  data  presentation  systems.  The  technology  is 
progressing  rapidly.  However,  there  are  a  number  of  problems  which  must  be 
resolved.  Including  the  effect  of  ambient  noise,  poor  recognition  rates,  and 
time  required  to  "train"  the  system  to  recognize  a  specific  user.  Touch  panel 
input  systems  are  more  nearly  ready  for  application  in  automated  technical 
data  presentation  systems.  The  primary  concern  is  their  durability  and 
ability  to  withstand  the  Inevitable  rough  treatment  that  the  system  will 
receive  in  the  maintenance  environment. 


Presentation  Formats 

Proceduralized  Data.  Oata  for  procedural  information  should  be  presented 
in  a  Job  Guide- like  step-by-step  format  (reference  MIL-M-83495) .  Three  basic 
types  of  Information  must  be  provided:  (a)  preparation  information  (input 
conditions);  (b)  procedures;  and  (c)  warnings,  cautions,  and  notes.  The 
following  format  recommendations  are  made  for  use  with  automated  technical 
data  presentation  systems: 


1.  Input  Conditions  Format.  The  input  conditions  frames  should  provide 
the  technician  with  all  of  the  information  that  is  needed  to  prepare  to 
perform  the  task.  This  information  should  include,  as  a  minimum,  applicable 
serial  numbers;  personnel  required;  parts  information;  supplies  required, 
equipment  conditions;  and  warnings,  cautions,  and  notes.  The  input  conditions 
should  provide  information  in  an  uncluttered,  list  format  similar  to  that 
illustrated  in  Figure  46. 

2.  Warnings,  Cautions,  and  Notes.  Warnings,  cautions,  and  notes  should 
be  presented  in  an  attention-getting  format.  Unique  banners  should  be  used  to 
Indicate  that  the  information  presented  is  a  warning,  caution,  or  note. 
Warnings,  cautions,  and  notes  may  be  presented  on  individual  frames  or,  for 
large  screens,  presented  on  a  frame  with  other  information.  When  the  latter 
approach  is  used,  care  must  be  taken  to  ensure  that  the  warning,  caution,  or 
note  stands  out  and  cannot  be  missed.  All  warnings,  cautions,  and  notes 
applicable  to  a  procedure  should  be  presented  at  the  beginning  of  the 
procedure  immediately  following  the  input  conditions  and  again  immediately 
prior  to  the  step  to  which  they  apply.  ' 

3.  Procedural  Data  Formats.  Procedural  data  should  be  presented  in  a 
step-by-step  format.  Data  Intended  for  use  by  inexperienced  personnel  should 
be  supported  by  illustrations  keyed  to  the  text.  Data  for  use  by  experienced 
personnel  may  or  may  not  be  supported  by  graphics,  depending  upon  the  nature 
of  the  data  and  the  anticipated  skills  of  the  experienced  user.  Normally, 
experienced  personnel  will  not  require  locator  diagrams.  However,  they  may 
require  diagrams  presenting  test  data  (such  as  expected  wave  patterns  on  an 
oscilloscope).  The  allocation  of  space  and  arrangement  of  graphics  and  text 
In  the  frame  are  dependent  upon  the  amount  of  textual  data,  number  and  size  of 
Illustrations,  and  the  size  and  shape  of  the  screen.  The  arrangement  of  the 
data  on  the  frame  should  be  uncluttered  and  easy  to  read.  See  Figures  44  and 
45  for  examples  of  typical  formats. 

Non-procedural 1 zed  Data  (text).  Textual  Information  (such  as  theory  of 
operation,  system  descriptions,  etc.)  should 'be  presented  In  a  blocked  text 
format.  Information  should  be  displayed  in  an  uncluttered,  easy-to-read 
arrangement.  Specific  layouts  are  dependent  upon  the  nature  of  the 
Information  to  be  presented. 

Large,  Complex  Diagrams.  Large,  complex  diagrams,  such  as  schematics  and 
wiring  diagrams,  present  a  special  problem  since  the  complete  diagram  cannot 
be  viewed  on  the  screen  at  one  time.  The  use  of  a  pyramiding  approach  such  as 
that  tested  in  the  Hughes/Rockwell  design  study  (Section  IV)  is  recommended. 
In  this  approach,  an  overall  diagram  Identifying  the  basic  functions  or 
functional  areas  of  the  underlying  diagram  Is  presented  first.  The  user  is 
able  to  Identify  the  function  of  Interest.  By  entering  an  Identifier  or  by 
moving  a  cursor  to  the  appropriate  block  on  the  diagram,  the  user  can  then 
call  up  the  needed  portion  of  the  diagram.  The  following  recommendations  are 
made  for  use  of  the  pyramiding  approach  for  the  presentation  of  large,  complex 
diagrams: 

1.  The  overall  functional  diagram  should  be  presented  at  a  size  which  1$ 
easily  readable  without  rescaling.  If  the  number  of  functions  to  be 
represented  are  too  large  for  presentation  on  the  screen,  multiple  levels  of 
functional  diagrams  may  be  used.  However,  the  number  of  multiple-level 
function  diagrams  should  be  limited  (preferably  no  more  than  3  or  4). 
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2.  After  the  user  has  made  a  selection  from  the  functional  diagram,  the 
portion  of  the  diagram  providing  detailed  information  on  the  function  selected 
should  be  presented  on  the  display.  The  scroll  function  should  be  active  so 
that  the  user  can  move  the  diagram  to  display  information  not  initially 
displayed.  The  scroll  function  should  allow  the  user  to  scroll  the  diagram 
into  the  next  functional  area  to  permit  tracing  of  signal  flows  and 
interrelationships  between  functional  areas. 

3.  Provision  should  be  made  for  ready  access  (from  the  detailed  diagram) 
to  the  functional  diagram  to  assist  the  users  in  maintaining  their 
orientation.  Access  may  be  provided  by  presenting  the  functional  diagram  in  a 
second  window  on  the  display  or  by  providing  rapid  switching  back  and  forth 
between  the  diagrams.  If  the  windowing  approach  is  used,  the  functional 
diagram  must  be  presented  at  a  size  that  permits  the  data  to  be  read  without 
enlargement.  Also,  when  windowing  is  used,  the  functional  diagram  window  must 
be  "active"  so  that  the  user  may  manipulate  the  diagram  or  select  a  different 
function  for  detailed  display. 

IPB  Data.  Two  basic  formats  are  recommended  for  IPB  data:  an  exploded 
view  format  and  a  parts  information  format. 

1.  Exploded  View  Format.  The  exploded  view  format  has  two  functions:  to 
Illustrate  the  components  of  an  assembly  and  their  interrelationships,  and  to 
serve  a$  a  graphic  index.  Multiple  levels  of  exploded  views  may  be  used  so 
that,  Kiere  appropriate,  an  exploded  view  of  the  next-level  subassembly  may  be 
recalled.  The  exploded  view  should  fill  the  full  frame.  Each  item  shown  on 
the  illustration  should  be  Identified  by  a  callout  which  can  be  entered  into 
the  system  to  recall  detailed  information  on  the  item.  In  addition,  an  option 
should  be  provided  to  recall  Information  on  the  assembly  as  a  whole  and,  where 
appropriate,  the  next  lower  subassembly. 

2.  Parts  Information  Format.  This  frame  should  provide  a  detailed 
illustration  of  the  part  In  one  section  of  the  frame  and  a  listing  of 
pertinent  Information  about  the  part  (part  number,  reference  designator,  stock 
number,  etc.)  in  another  window.  The  frame  should  provide  the  option  to  go  to 
the  exploded  view  of  the  appropriate  assembly  or  subassembly.  See  Figure  41 
for  an  example  parts  Information  frame. 

Common  Format  Elements.  All  formats  must  contain  the  following  elements: 

1.  Header.  The  header  should  present  the  TO  number,  procedure  title,  and 
a  frame  or  other  unique  identifier.  Other  identifiers  such  as  a  MIDAS  code 
may  also  be  included  if  desired.  This  information  should  always  be  presented 
at  the  top  of  the  frame. 

2.  Footer.  The  footer  should  be  used  to  identify  active  function  keys 
and  to  present  system  messages  (such  as  "NEXT  not  active").  Function 
Identifications  should  always  appear  on  the  last  line  of  the  display.  System 
messages  may  be  on  the  last  line  (writing  over  the  function  identifications) 
or  on  a  line  above  the  function  Identifications  as  appropriate. 

Other  Format  Considerations.  Consideration  should  be  given  to  the 
following  factors  in  developing  formats  for  the  presentation  of  technical  data 
on  an  automated  system: 
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1.  Font  size.  The  font  used  to  present  procedural  data  should  be 
sufficiently  large  to  be  easily  read  at  a  distance  of  6  to  8  feet.  A  smaller 
font  (readable  at  2  to  3  feet)  may  be  used  for  support  information  (such  as 
theory  of  operation)  which  is  normally  read  at  a  closer  distance.  (Note: 
character  heights  of  .250  and  .125  inch  are  required  for  readability  at  6  feet 
and  3  feet,  respectively.) 

2.  Font  Style.  The  font  style  selected  should  be  clear  and  easy  to 
read.  Letters  which  are  similar  in  shape  (such  as  a,  e,  and  o)  should  be 
easily  distinguishable. 

3.  Case.  A  combination  of  standard  lowercase  and  uppercase  characters 
should  be  used.  The  upper/lowercase  combination  provides  more  information  and 
is  easier  to  read. 

4.  Font  Generation.  Preconstructed  or  "built-in"  fonts  should  be 
provided  for  all  fonts  frequently  used  to  present  data  to  speed  presentation 
of  textual  information.  A  requirement  to  "draw"  each  letter  significantly 
increases  the  time  required  to  present  a  frame  of  data. 


Hardware  Considerations 

Hardware  for  shop  and  portable  systems  must  perform  essentially  the  same 
functions.  However,  the  constraints  for  each  environment  present  different 
problems  for  the  designers  of  systems  for  each  application.  The  following 
factors  should  be  considered  in  designing  hardware  for  automated  technical 
data  presentation  systems  for  shop  and  flightline  use: 

Shop  Systems.  The  constraints  imposed  by  the  shop  environment  provide 
fewer  restrictions  than  the  flightline  environment  in  terms  of  physical  size, 
power  consumption,  and  environmental  conditions  (temperature,  humidity, 
lighting,  etc.).  There  are  several  tradeoffs  to  be  considered  in  selecting 
hardware  for  a  shop  system.  The  following  recommendations  are  provided: 

1.  Physical  Characteristics.  Limiting  the  physical  size  of  shop  systems 
is  not  as  critical  as  for  portable  units.  However,  it  is  highly  desirable 
that  the  hardware  be  kept  as  small  as  possible  so  that  it  does  not  crowd  the 
technician's  work  space.  The  display  should  be  as  compact  as  possible.  The 
display  unit  itself  should  fit  conveniently  on  the  workbench  where  it  can  be 
easily  viewed  from  all  work  areas  of  the  bench  (30-degree  viewing  angle).  The 
display  unit  should  be  as  compact  as  possible.  The  hardware  required  to 
support  the  display  (processors,  graphics  boards,  etc.)  should  be  located  in  a 
separate  unit  (located  somewhere  off  the  workbench)  so  that  the  display  unit 
is  composed  only  of  the  display  itself  and  its  mounting. 

Other  physical  units  of  the  system  (processor,  secondary  memory, 
interfaces,  etc.)  should  be  located  somewhere  off  the  workbench.  The  exact 
location  of  these  components  is  dependent  upon  the  overall  configuration  of 
the  automated  technical  data  system  (which  may  use  dedicated  units  for  each 
workbench  or  use  a  distributed  system  with  a  central  processor  supporting 
several  shops  or  possibly  the  whole  base)  and  is  beyond  the  scope  of  this 
project.  An  option  that  should  be  considered  for  some  applications  is  placing 
the  complete  unit  (display,  secondary  memory,  etc.)  on  a  special  cart  (such  as 


that  used  for  an  oscilloscope)  which  can  be  moved  In  and  about  the  shop  as 
necessary. 

For  operational  applications,  the  hardware  should  be  sufficiently 
ruggedlzed  to  support  operations  In  worldwide  deployments  or  combat  actions. 
The  degree  of  ruggedlzatlon  required  for  deployable  test  equipment  should  be 
sufficient  for  most  applications.  Also,  it  should  be  remembered  that 
deployment  requirements  may  add  other  constraints  (power  availability, 
packaging,  etc.)  which  must  be  considered  in  system  design. 

2.  Display.  The  screen  must  be  of  sufficient  size  to  permit  presentation 
of  a  meaningful  amount  of  data  in  a  readable  form.  It  must  be  large  enough  to 
permit  the  use  of  a  font  that  is  easily  readable  and  the  presentation  of 
graphics  in  sufficient  detail  for  easy  use.  It  must  be  remembered  that,  in 
many  cases,  the  display  must  be  read  from  as  far  away  as  6  to  8  feet. 
Experience  with  the  CMAS  I  and  CMAS  II  and  similar  systems  suggests  that  a 
display  size  of  approximately  5x8  inches  is  adequate  for  most  shop 
applications. 

The  display  must  have  adequate  resolution  to  permit  display  of  relatively 
complex  graphics.  There  are  no  firm  standards  for  display  resolution  for 
technical  data  presentation.  The  display  should  have  a  medium  level  of 
resolution.  Experience  with  the  prototypes  developed  in  this  project 
indicates  that  the  display  should  have  a  minimum  of  65  pixels  per  inch.  This 
level  of  resolution  was  found  to  be  adequate  for  most  graphics  found  in 
technical  data. 

The  display  should  be  easy  to  read  under  a  variety  of  lighting 
conditions.  To  meet  this  requirement,  it  must  have  some  provision  to  limit 
glare  (through  the  use  of  a  filter,  shade,  or  repositioning);  have  a  high 
contrast  ratio;  and  have  a  large  angle  of  view  (at  least  30  degrees)  to  permit 
the  user  to  read  from  all  work  positions  without  repositioning  the  display. 

3.  Memory  devices.  The  system  should  provide  a  hard  disk  memory  device 
with  sufficient  capacity  to  maintain  the  data  for  all  equipment  maintained 
using  the  workbench.  The  amount  of  memory  required  is  a  function  of  the 
amount  of  information  to  be  presented  and  the  manner  in  which  the  data  are 
prepared  (coding  scheme  used,  data  compaction  techniques  used,  etc.). 

Portable  System.  Creative  application  of  state-of-the-art  technology  is 
necessary  to  design  a  portable  computer  system  suitable  for  the  presentation 
of  technical  data  in  the  flightline  environment.  The  current  state  of  the  art 
will  not  support  the  development  of  the  "ideal"  portable  system.  Significant 
advances  are  needed  to  improve  memory  capacities;  improve  the  size, 
resolution,  and  viewability  of  flat-panel  displays  (while,  limiting  power 
consumption);  improve  processor  speed;  improve  graphics  generation  and  display 
capabilites;  and  increase  battery  capacity.  However,  the  technology  will 
support  the  development  of  portable  systems  which  are  satisfactory 
first-generation  portable  computer  systems  for  operational  use.  The  following 
recommendations  should  be  considered  in  the  development  of  such  a  system: 

1.  Physical  characteristics.  The  system  must  be  small  enough  and  light 
enough  for  convenient  use.  A  female  technician  qualifying  for  a  maintenance 
career  field  should  be  able  to  pick  up  the  unit  (by  the  body,  not  the  handle) 
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with  one  hand  and  hold  it  with  one  hand  for  at  least  5  minutes.  As  design 
goals,  it  is  recommended  that  the  next-generation  portable  system  not  exceed  6 
pounds  in  weight,  and  not  be  more  than  2  inches  thick,  10  inches  wide,  and  12 
inches  long. 

The  unit  must  be  sufficiently  ruggedized  to  withstand  the  rigors  of 
flightline  use  and  the  most  stringent  environmental  conditions  expected  to  be 
encountered. 

2.  Display.  The  basic  display  requirements  described  above  for  the 
shop- level  system  are  also  applicable  for  the  portable  system.  However,  the 
constraints  imposed  on  the  portable  system  make  the  selection  of  a  display  a 
difficult  problem.  Additional  factors  which  must  be  considered  are  described 
in  the  following  paragraphs: 

a.  The  power  consumption  must  be  limited  due  to  the  requirement  to 
operate  on  battery  power.  Power  consumption  and  battery  capacity  must  provide 
for  the  computer  to  operate  without  recharging  the  batteries  for  at  least  4 
hours  and  preferably,  8  hours  or  longer. 

b.  The  display  must  be  suitable  for  viewing  under  difficult  lighting 
conditions— ranging  from  near  darkness  to  bright  sunlight. 

c.  The  display  must  be  able  to  withstand  very  rigorous  handling, 
including  dropping  from  a  height  of  10  feet. 

d.  The  display  must  be  sufficiently  thin  to  allow  it  to  fit  In  the 
limited  space  available  on  the  portable  unit. 

The  above  constraints  limit  the  display  to  a  flat-panel  technology.  At 
the  present  time,  none  of  the  flat-panel  technologies  is  capable  of  providing 
a  completely  suitable  display.  However,  rapid  advances  are  being  made  in  this 
technology  area  which  may  resolve  the  problem  before  a  portable  system  Is 
available  for  operational  use. 

3.  Memory.  The  portable  unit  must  have  a  secondary  memory  capable  of 
providing  the  technician  with  all  of  the  information  needed  to  complete  an 
assigned  task  without  returning  to  the  shop  to  "reload."  The  memory  cartridge 
approach  adopted  for  the  PCMAS  appears  to  be  a  satisfactory  solution  to  this 
problem;  however,  it  has  not  been  fully  tested.  Future  advances  in  memory 
technology  may  make  It  possible  to  store  the  required  data  for  an  entire 
system  on  board  the  portable  unit  without  the  necessity  to  rely  on 
interchangeable  cartridges. 

4.  Ruggedization.  The  portable  system  must  be  rugged  enough  to  withstand 
the  very  rough  treatment  that  can  be  expected  In  the  flightline  environment. 
It  must  be  able  to  withstand  extremes  of  temperature  and  humidity,  rain  and 
liquid  spills,  being  dropped  from  several  feet  and  various  other  shocks,  and 
the  effects  of  high-altitude  operation.  In  addition,  it  must  be  shielded  to 
prevent  electromagnetic  Interference. 
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Software  Considerations 


The  following  software  features  should  be  Included  In  any  operational 
system  developed: 

Shoo/Portable  Systems  Software  Compatibility.  To  the  maximum  extent 
possible,  the  same  software  should  be  used  for  the  shop  and  portable  systems. 
The  only  differences  In  the  software  should  be  those  necessary  to  accommodate: 

1.  Unique  features  or  capabilities  of  each  system  (e.g.,  housekeeping 
functions  for  the  shop  system  and  Interactive  diagnostic  functions  for  the 
portable  system). 

2.  Characteristics  unique  to  each  system  (e.g.,  different  display  and 
memory  devices). 


Software  Capabilities.  The  following  types  of  software  should  be  provided 
for  the  shop  and  portable  systems. 

1.  Operating  System.  A  standard  operating  system  (such  as  Unix)  should 
be  used.  The  same  operating  system  should  be  used  for  the  shop  and  portable 
systems.  The  complete  operating  system  should  be  provided  for  the  shop 
system.  The  operating  system  for  the  portable  system  should  be  a  "pared-down" 
version  containing  only  those  features  required  to  support  the  system's 
designated  functions. 

2.  Add 1 1 cat 1 ons  Software .  As  a  minimum,  three  types  of  applications 
software  should  he  provided: 

a.  Data  Presentation  Software.  This  software  should  provide  for  the 
display  of  technical  data  Including  maintenance  procedures,  troubleshooting 
procedures  (non- automated  diagnostic  routines),  IPB  Information,  and  support 
Information  (theory  of  operation,  system  descriptions,  etc.).  The  same  basic 
software  should  be  provided  for  both  shop  and  portable  systems.  The  software 
should  be  designed  to  operate  from  the  same  data  base  so  that  shop-level 
technical  data  may  be  displayed  on  the  portable  system  and  vice-versa. 

b.  Interactive  Diagnostics  Software.  This  software  should  be 
provided  for  diagnostic  procedures  which  interact  with  aircraft  systems.  It 
is  anticipated  that  this  capability  will  be  provided  with  portable  systems 
only.  However,  future  efforts  may  extend  the  techniques  to  Interface  shop 
systems  to  test  equipment.  In  that  event,  the  shop  system  will  Include 
diagnostics  software. 

c.  Maintenance  Data  Collection  (MDC)  System.  This  software  provides 
a  means  for  the  user  to  record  and  input  Information  on  maintenance  actions 
completed  to  the  MDC  system.  This  software  should  be  provided  for  both 
systems.  (Note:  Depending  upon  system  design,  MDC  data  may  be  transferred 
from  the  portable  system  to  the  MDC  system  via  the  shop  system.) 

The  technical  data  presentation.  Interactive  diagnostics,  and  MDC  system 
software  must  be  fully  integrated  so  that  they  appear  as  one  system  to  the 
user. 
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Other  Cons 1 derations 


Use  of  Color.  Color  can  be  useful  to  emphasize  critical  Information  (such 
as  warnings)  and  to  color-code  other  types  of  Information  (thereby  providing 
additional  Information).  However,  there  Is  little.  If  any,  empirical  data  to 
suggest  that  the  use  of  color  for  automated  technical  data  presentation 
systems  Improves  performance.  Other  applications  of  color  for  Information 
systems  suggest  that  the  use  of  color  for  the  presentation  of  technical  data 
may  be  beneficial  If  properly  applied.  However,  the  ability  to  present  data 
In  color  should  not  be  a  major  criterion  for  selection  of  the  hardware  for 
such  a  system.  For  the  Initial  system.  It  Is  more  Important  to  use  a  display 
that  Is  large  enough  to  present  a  meaningful  amount  of  data  in  a  readable  size 
and  with  sufficient  resolution.  Additional  research  Is  needed  to  assess  the 
value  of  color  for  the  presentation  of  technical  data. 

Orientation.  It  Is  Important  that  the  users  know  where  they  are  In  the 
data  base.  The  system  should  provide  visual  cues  to  help  the  users  maintain 
their  orientation.  The  primary  tool  available  for  this  purpose  Is  the  header 
at  the  top  of  each  frame.  As  a  minimum,  the  header  should  provide  the 
following  Information  to  the  user:  which  TO  Is  being  used,  the  name  of  the 
procedure,  an  Indicator  showing  the  data  track,  and  some  step  or  frame 
Identifier.  In  addition,  an  Indicator  should  be  provided  to  tell  the  user  the 
relative  position  in  the  procedure  (e.g.,  step  5  of  34). 

Help.  The  system  should  provide  ready  access  to  a  help  function  which 
provides  guidance  on  the  use  of  the  system.  The  help  feature  should  provide 
the  following  types  of  Information: 

1.  Situation  specific.  Information  on  the  current  situation  and  the 
available  options  (this  Is  where  you  are,  and  here  are  your  options). 

2.  System  Operation.  An  on-line  Instruction  manual  on  how  to  use  the 
system. 

The  help  function  should  be  accessed  through  a  dedicated  help  key  or 
through  the  main  menu.  The  use  of  a  dedicated  help  function  key  Is  desirable 
If  the  hardware  provides  sufficient  function  keys  to  allow  one  key  to  be  set 
aside  for  this  purpose  and  still  satisfy  requirements  for  other  essential 
functions.  - 


Research  Needs 


As  Indicated  earlier,  many  of  the  design  features  of  the  CMAS  II  were 
based  upon  analysis  and  the  developer's  "best  guess."  Although  the  system 
based  upon  these  best  guesses  has  proven  to  be  successful.  It  Is  likely  that 
some  of  the  features  Incorporated  in  the  system  are  not  optimal.  A  better 
understanding  of  the  features  and  definition  of  requirements  will  result  In 
the  development  of  even  more  effective  systems.  Design  features  and  design 
Issues  which  require  additional  research  are  discussed  below. 


Graphics  Issues 

Graphics  are  critical  elements  of  an  automated  technical  data 
presentation.  Several  graphics  Issues  require  study. 

Graphics  Complexity.  The  issue  of  how  complex  graphics  should  be  for 
automated  technical  data  presentation  Is  not  well  understood.  Two  factors  are 
involved:  usability  and  storage  requirements.  The  graphic  must  contain 
enough  detail  to  permit  the  user  to  Identify  the  Items  of  interest,  but  must 
not  be  so  detailed  that  the  Item  of  Interest  Is  lost  In  a  blur  of  lines.  In 
addition,  as  a  practical  necessity.  It  Is  essential  to  keep  graphics 
complexity  to  a  minimum  to  reduce  storage  requirements.  Preliminary  findings 
Indicate  that  much  of  the  detail  can  be  omitted  from  an  Illustration  without  a 
decrement  In  performance.  Providing  only  limited  cues  seems  to  be  sufficient 
In  many  cases  to  permit  Identification  of  the  Item  of  Interest.  Additional 
research  Is  needed  for  a  variety  of  applications  to  determine  how  complex 
graphics  must  be  In  order  to  support  maintenance.  Guidelines  must  then  be 
developed  to  guide  the  technical  data  writer/illustrator  to  ensure  that  the 
appropriate  level  of  detail  Is  provided  in  all  graphics  and  to  ensure 
consistency  of  application  across  the  data  base. 

Use  of  Graphics  In  Procedural  Data.  There  Is  general  agreement  that,  at 
least  for  Inexperienced  personnel,  graphics  should  be  provided  to  support 
procedural  data.  Two  Issues  require  study:  (a)  arrangement  of  text  and 
graphics,  and  (b)  the  form  of  callouts  used.  One  of  the  Hughes  design  studies 
examined  the  approach  of  Integrating  text  and  graphics  (see  page  67). 
Although  the  study  Indicated  that  the  approach  has  promise.  It  was  not 
conclusive.  Additional  studies  of  this  and  other  approaches  to  arranging  text 
and  graphics  are  needed. 

The  concept  of  using  callouts  to  key  text  to  Illustrations  for  procedural 
data  has  been  demonstrated  to  be  effective  for  paper-based  systems  (e.g.,  Job 
Guide  manuals).  In  addition.  It  was  successfully  applied  in  the  CMAS.  There 
are  at  least  two  ways  of  keying  callouts  to  the  text.  One  approach  Is  to  use 
numbers  In  parentheses  Immediately  following  the  Item  of  Interest  In  the  text 
which  correspond  to  a  numbered  arrow  pointing  to  the  Item  of  Interest  In  the 
Illustration.  Another  approach  Is  to  use  the  number  of  the  procedural  step  to 
refer  to  a  numbered  and  labeled  arrow  pointing  to  the  1tem(s)  of  Interest  in 
the  Illustration.  Research  Is  needed  to  evaluate  the  relative  effectiveness 
of  the  two  approaches  and  any  other  approaches  which  may  be  developed. 

Use  of  Zoom/Scroll  Capabilities.  It  Is  generally  believed  that  an 
automated  technical  data  presentation  system  should  provide  the  capability  to 
zoom  and  scroll  selected  graphics.  Research  is  needed:  (a)  to  determine  the 
most  effective  techniques  for  control  Una  the  zoom  and  scroll 'features  (e.g., 
arrow  keys,  joystick,  mouse,  etc.);  (b)  to  establish  the  most  appropriate 
scaling  factors;  and  (c)  to  develop  guidelines  for  specifying  when  the 
capabilities  will  be  made  available  and  for  specifying  the  limits  placed  on 
their  use. 


Presentation  of  Schematics.  The  studies  conducted  under  this  project  have 
shown  that  schematic  and  other  large  diagrams  presented  on  a  computer  display 
can  be  used  effectively,  and  have  provided  some  suggestions  on  how  they  can  be 
designed  for  more  effective  use.  Potential  techniques  for  Improving  the  use 


of  schematic  and  other  large  diagrams  include:  (a)  the  use  of  pyramiding  (see 
page  61);  (b)  the  use  of  overview  or  function  diagrams,  (c)  the  highlighting 
of  signal  paths;  and  (d)  computer-generated  signal  tracing  (i.e.,  the  user 
Identifies  a  component  and  the  coaputer  Identifies  the  coaponents  that  that 
coaponent  Is  dependent  upon). 

Pseudoaniaation.  In  the  initial  feasibility  study,  Frazier  proposed  the 
use  of  pseudoaniaation  to  present  procedural  Instructions  for  selected  tasks 
(see  page  21).  The  concept  was  not  pursued  at  the  tiae  because  it  was  consi¬ 
dered  impractical  with  the  state  of  the  art  in  coaputer  anlaation  at  the 
tiae.  However,  significant  advances  have  been  aade  in  anlaation  technology 
which  aay  aake  the  technique  worthy  of  further  investigation. 


Display  Issues 

The  following  display  Issues  require  study: 

Resolution.  The  selection  of  an  appropriate  level  of  resolution  for  the 
display  of  ah  autoaated  technical  data  presentation  systea  Involves  two 
tradeoffs:  cost  and  speed.  The  higher  the  resolution,  the  greater  the  cost 
of  the  display.  The  higher  the  resolution,  the  aore  coaputations  that  aust  be 
aade  and  hence,  the  aore  tiae  required  to  display  a  graphic.  The  optlaua 
resolution  for  autoaated  technical  data  systea  displays  is  not  known.  The 
CMAS  studies  Indicated  that  the  level  of  detail  of  the  Grid  Compass 
electroluminescence  display  (66  pixels  per  Inch)  Is  adequate  for  presenting 
technical  data  for  electronic  systems.  However,  electronic  systems  require 
relatively  simple  graphics.  It  Is  uncertain  whether  the  same  level  of 
resolution  will  be  adequate  for  the  aore  complex  graphics  found  In  mechanical 
systems. 

Display  Screen  Size.  The  optimum  screen  size  for  displays  for  In-shop  and 
portable  technical  data  presentation  systems  Is  not  known.  Screen  size  Is  not 
seen  as  a  major  problem  for  In-shop  systems  since  space  Is  not  a  major 
constraint  (although  there  Is  likely  an  optimum  size  beyond  which  the 
advantages  do  not  justify  the  cost  or  space  usage  associated  with  a  larger 
screen).  However,  screen  size  Is  a  major  consideration  In  the  development  of 
a  portable  system,  since  It  Is  a  major  determinant  of  the  size  of  the  unit 
Itself.  Preliminary  test  results  (Dwyer,  1985)  Indicate  that  a  number  of 
factors  (Including  graphics  Information  density,  dlscrlmlnablllty  of  elements 
of  a  graphic,  and  display  resolution)  Influence  screen  size  requirements. 
Research  1$  needed  to  Identify  the  optimum  size  of  a  screen  for  portable  and 
In-shop  systems. 

Use  of  Color.  As  Indicated,  the  Impact  of  the  use  of  color  for  automated 
technical  data  presentation  systems  Is  not  well  understood.  Research  Is 
needed  to  identify  techniques  to  use  color  to  Improve  performance  when  using 
an  automated  system.  Once  such  techniques  are  developed,  research  Is  needed 
to  evaluate  their  effect  on  performance  to  provide  a  basis  for  determining  If 
the  improvement  In  performance  justifies  the  additional  cost. 

Other  variables.  There  are  a  number  of  other  factors  (such  as  contrast 
ratios,  brightness,  font  size,  etc.)  which  influence  performance  when  using  a 
computer  display.  There  are  large  amounts  of  data  In  the  literature  on 
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variables  of  this  sort.  However,  aost  of  the  data  have  been  developed  under 
working  conditions  which  are  very  different  from  those  encountered  In  the 
aalntenance  environment.  Thus,  the  data  cannot  be  applied  with  confidence  to 
the  developaent  of  coaputer  displays  for  use  In  the  aalntenance  envlronaent. 
Research  Is  needed  to  evaluate  each  of  these  variables  under  conditions 
slallar  to  those  encountered  In  the  aalntenance  envlronaent. 


Technical  Data  Presentation  Considerations 


Multiple  Levels  of  Detail.  Although  the  aultlple  level  of  detail  or  track 
concept  was  well  received  by  the  users  and  Is  believed  to  be  an  effective 
technique,  there  are  several  Issues  which  require  further  study.  A  major 
question  Is  how  aany  levels  of  detail  should  be  provided  and  how  they  should 
be  defined.  Experience  In  both  CMAS  efforts  Indicated  that  two  tracks  of  data 
are  sufficient  for  the  Interaedlate  level  aalntenance  of  avionics  equlpaent. 
Because  many  of  these  tasks  are  relatively  straightforward.  It  Is  difficult  to 
provide  enrichment  to  aid  the  novice  (e.g..  It  Is  difficult  to  provide 
enrichment  for  a  step  such  as  "Turn  MODE  SELECT  SWITCH  to  MODE  AM).  However, 
two  levels  of  detail  may  not  be  sufficient  for  all  types  of  aalntenance.  Some 
mechanical  tasks,  for  example,  may  require  three  tracks  to  provide  adequate 
Information  to  the  novice.  Also,  It  may  be  that  soae  portions  of  a  procedure 
may  require  a  third  track  while  others  require  only  one  or  two  tracks. 
Research  is  needed  to  determine  how  multiple  tracks  are  to  be  used,  to  define 
the  multiple  tracks  to  be  used  In  automated  technical  data  presentation 
systems,  and  to  define  the  applications  for  which  two  or  more  tracks  are 
appropriate. 

A  related  problem  Is  the  question  of  how  auch  detail  aust  be  provided  In 
the  minimum  level  of  detail  track  (Track  1).  The  original  concept  was  that 
this  track  would  be  for  the  very  experienced  technician  who  needed  only  a 
reminder  of  the  basic  subtasks  which  (It  was  assumed)  he  would  know  how  to 
perform.  (He  could  refer  to  the  Track  2  procedure  If  he  so  desired..) 
However,  In  application,  the  question  arises  as  to  whether  this  level  of 
detail  Is  sufficient  to  satisfy  safety  and  quality  assurance  requirements. 
Early  Indications  are  that,  although  very  acceptable  to  experienced 
technicians,  this  level  of  detail  may  not  be  acceptable  to  safety  and  quality 
assurance  officials.  Research  Is  needed  to  evaluate  this  problem  and 
determine  the  appropriate  minimum  level  of  detail  for  Track  1. 

Steps  per  Screen.  Depending  upon  the  amount  of  Information  to  be 
presented,  the  size  and  number  of  graphics,  and  the  size  of  the  screen.  It  Is 
often  possible  to  put  more  than  one  step  on  a  frame  of  procedural  data. 
Presenting  as  many  steps  as  possible  on  a  screen  has  the  advantage  of  reducing 
the  number  of  times  the  technician  has  to  stop  and  press  the  NEXT  key.  It 
also  makes  full  use  of  the  screen  and  avoids  the  possible  complaint  from  the 
technician  that  the  space  could  have  been  used  to  present  the  next  step  so 
that  he  would  not  have  to  stop  and  press  NEXT.  However,  presenting  only  one 
step  at  a  time  has  the  advantage  of  forcing  the  technician  to  acknowledge  each 
step  and  reducing  the  chance  that  he  may  Inadvertently  skip  a  step.  A 
compromise  approach  Is  to  present  multiple  steps  per  screen  and  highlight  each 
step  as  It  Is  performed.  In  this  case,  the  technician  Is  required  to  indicate 
to  the  system  when  the  step  has  been  completed.  Research  Is  needed  to 
determine  which  approach  Is  the  most  desirable. 
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Presentation  of  Diagnostics  Information.  The  interactive  diagnostic 
techniques  being  developed  under  EKe  mOAs  program,  and  their  planned 
Integration  with  the  automated  technical  data  presentation  system,  raise  a 
number  of  Issues  which  have  not  been  adequately  studied.  Some  questions  which 
must  be  studied  are: 

1.  What  Is  to  be  the  role  of  the  technician  in  an  Interactive  diagnostics 
system?  How  much  of  the  diagnostic  process  should  be  performed  by  the 
computer  and  how  much  by  the  technician?  How  much  control  should  the 
technician  have  (l.e.,  should  he  be  permitted  to  select  tests  to  be  performed)? 

2.  For  what  experience  level  of  technician  should  the  system  be 
designed?  Should  the  system  be  suitable  for  both  novice  and  experienced 
personnel?  If  so,  what  should  be  the  differences  In  the  way  information  Is 
presented  and  the  control  provided  to  the  technician?  How  much  system 
knowledge  can  be  assumed  (e.g..  Is  the  technician  a  highly  trained  specialist 
or  a  generalist)? 

3.  How  should  the  Interactive  diagnostics  system  be  Integrated  with  the 
technical  data  system?  Should  It  be  considered  part  of  the  technical  data  and 
be  transparent  to  the  user?  Should  It  be  a  completely  separate  system  which 
calls  up  technical  data  routines  as  needed?  Mho  maintains  the  Interactive 
diagnostics  data  (technical  data  personnel  or  diagnostics  engineers)? 

Pool  Information.  There  Is  general  agreement  that  the  concept  providing 
direct  access  to  selected  support  or  "pool*  Information  (such  as  theory  of 
operation  or  parts  Information)  from  a  procedure  Is  desirable.  However,  the 
question  of  what  Information  and  how  much  Information  has  not  been  adequately 
answered.  As  originally  proposed  by  Frazier  (see  page  15),  the  pool 
Information  was  to  Include  Information  (such  as  how  to  use  test  equipment) 
from  outside  of  the  basic  technical  order.  This  Is  a  desirable  concept. 
However,  there  are  practical  limitations  which  must  be  considered.  The 
presentation  of  pool  Information  other  than  the  basic  data  procured  with  all 
weapon  systems  can  be  cost  effective  only  If  the  pool  Information  Is  already 
part  of  the  technical  data  (which  training  data  may  not  be).  Another  limiting 
factor  Is  storage  space,  especially  for  portable  units.  Large  f1xed>base 
systems  may  be  able  to  handle  all  of  the  information  desired.  However,  at 
least  until  much  larger  capacity  data  storage  devices  are  available.  It  will 
be  necessary  to  limit  access  to  pool  Information  for  portable  systems. 
Research  Is  needed  to  define  what  types  of  pool  Information  are  essential  and 
cost  effective  and  to  develop  criteria  for  determining  what  pool  Information 
Is  to  be  made  available. 


Man/Machine  Interface  Issues 

Function  Key  Use.  Further  research  Is  needed  to  determine  how  function 
keys  are  used.  Answers  are  needed  to  the  following  questions: 

1.  How  many  function  keys  should  be  hard-wired  and  how  many  should  be 
soft  keys? 

2.  Which  functions  should  be  Implemented  with  hard-wired  function  keys 
and  which  should  be  Implemented  with  soft  keys? 
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3.  In  some  applications.  It  may  be  necessary  to  have  more  function  keys 
active  than  are  available  on  the  keyboard.  What  Is  the  most  effective 
technique  for  handling  this  problea? 

4.  What  techniques  should  be  provided  to  help  the  user  recover  If  the 
wrong  function  key  Is  hit  accidentally? 

Use  of  Touch  Panel  Technology.  The  use  of  a  touch  panel  has  significant 
potential  as  a  user  Input  device.  The  prlaary  concern  with  the  use  of  touch 
panels  for  autoaated  technical  data  systeas  has  been  durability.  More  durable 
touch  panels  are  now  becoalng  available.  Research  Is  needed  to  test  and 
evaluate  these  panels  for  autoaated  technical  data  systea  applications. 

Use  of  Voice  Recognition.  Voice  recognition  has  the  potential  to  be  very 
useful  as  an  input  device  since  It  does  not  require  the  technicians  to  stop 
what  they  are  doing,  put  down  tools,  and  press  keys  to  obtain  the  desired 
Information.  Significant  advances  are  being  aade  In  voice  Input  technology. 
Research  Is  needed  to  test  and  evaluate  these  technologies  for  technical  data 
applications. 

Use  of  Voice  Synthesis.  The  use  of  voice  synthesis  to  present  technical 
Information  may  be  valuable  In  those  situations  where  It  Is  difficult  to 
locate  the  display  where  It  can  be  easily  viewed  by  the  technician.  It  may 
also  have  an  advantage  for  some  technicians  with  limited  reading  skills.  As 
with  voice  recognition,  significant  progress  has  been  made  In  developing  voice 
synthesis  technology.  Research  Is  needed  to  test  and  evaluate  these 
technologies  for  technical  data  applications. 


Data  Base  Issues 

Although  the  current  project  did  not  directly  address  the  data  base 
problem,  several  data  base  problems  have  been  Identified. 

Neutral  Data  Base.  The  concept  of  a  neutral  data  base  has  much  Inherent 
appeal. Although  the  neutral  data  bases  have  been  developed  with  some  success 
for  printing  applications,  the  concept  has  not  been  successfully  demonstrated 
for  use  with  automated  technical  data  systems.  Research  Is  needed  to  test  the 
concept  and  to  Identify  Its  limitations. 

Data  Base  Structure.  Previous  work  has  based  the  design  of  automated 
technical  data  bases  on  the  design  of  current  paper-based  technical  data. 
Thus,  the  data  bases  "mirror"  the  TO  system.  There  are  sections  for  theory  of 
operation,  system  description,  repair  procedures,  etc.— just  as  In  the  TO. 
There  Is  a  need  to  avoid  the  tendency  to  duplicate  the  TO  system  and  develop  a 
data  design  and  a  data  base  approach  that  Is  specifically  designed  for 
electronic  applications.  The  primary  concern  should  be  whether  all  of  the 
required  Information  Is  In  the  data  base  and  can  be  easily  retrieved. 

Portable  Svstem  Memory  Limitations.  For  at  least  the  near  future.  It  will 
not  be  possible  to  maintain  the  entire  data  base  on  one  portable  computer 
memory  device.  There  are  two  basic  approaches  to  resolving  this  problem:  (a) 
divide  the  data  base  Into  smaller  units  and  preload  the  data  on  several  memory 
devices,  or  (b)  load  a  memory  device  from  a  central  data  base  with  the 
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specific  Information  required  for  the  task  to  be  performed.  Each  approach  has 
advantages  and  disadvantages.  Research  Is  needed  to  evaluate  the  relative 
merits  of  each  approach.  In  addition,  each  approach  has  several  research 
Issues  which  must  be  resolved  before  It  can  be  effectively  applied.  The 
primary  Issue  to  be  resolved  for  the  preloading  approach  Is  how  to  partition 
the  data  base  In  a  meaningful  manner  that  will  avoid  duplication  of  data  while 
limiting  the  number  of  cartridges  that  the  technician  must  take  to  the  job. 
The  primary  Issue  to  be  resolved  for  the  custom  loading  approach  Is  how  to 
determine  what  Information  mist  be  loaded  so  that  the  technician  does  not  get 
to  the  job  and  find  that  the  cartridge  does  not  contain  some  essential 
Information. 


Other  Issues 


Portable  System  Use.  Very  little  Is  known  about  the  problems  which  will 
be  encountered  In  using  a  portable  automated  technical  data  system  on  the 
fllghtllne  or  a  similar  environment.  *  There  are  a  number  of  questions  which 
require  answers: 

1.  How  will  the  portable  system  be  used  In  the  field? 

2.  What  Is  the  maximum  distance  from  which  the  portable  system  display 
must  be  viewed?  What  problems  are  encountered  In  viewing  the  system? 

3.  Where  should  the  system  be  placed  for  convenient  use? 

4.  What  lighting  problem  are  encountered  In  using  the  system  on  the 
fllghtllne?  What  are  the  limits  of  the  lighting  conditions  under  which  the 
system  will  be  used? 

5.  What  are  the  maximum  weight  and  maximum  size  acceptable  for 
fllghtllne  use? 

6.  What  are  the  actual  power  constraints?  How  often  must  the  system 
operate  on  Its  own  batteries?  What  Is  the  minimum  battery  life  required  for 
operational  use  of  the  system  on  the  fllghtllne? 

7.  What  problems  are  encountered  In  using  memory  cartridges  on  the 
fllghtllne? 

8.  What  features  should  be  added  to  the  portable  system? 

9.  Are  there  any  tasks  that  cannot  be  supported  by  the  portable  system? 

10.  Do  technicians  like  to  use  the  portable  system  on  the  fllghtllne? 

11.  How  should  the  portable  system  be  used  to  support  multiperson  tasks? 

12.  What  problems  will  be  encountered  In  using  voice  recognition  on  the 
fllghtllne? 

Quality  Control.  Technical  data  for  presentation  on  an  automated  system 
must  be  i OOX  accurate.  Research  Is  needed  to  develop  quality  control 
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procedures  to  ensure  that  the  required  accuracy  Is  achieved.  The  primary  area 
requiring  research  Is  the  development  of  effective,  yet  practical,  techniques 
to  verify  the  accuracy  of  branching  Instructions  and  other  coding  Instructions 
which  control  the  order  of  presentation  of  the  data.  These  Instructions  must 
be  completely  accurate  to  ensure  that  the  user's  request  takes  him  where  he 
wants  to  go  and  that  screens  composed  of  elements  from  different  parts  of  the 
data  base  contain  the  data  that  they  are  supposed  to  contain  (e.g.,  the 
correct  text  and  graphics  are  linked  together). 

Comprehensive  Evaluation.  A  comprehensive  evaluation  In  which  a  complete 
set  of  data  for  an  operational  system  Is  tested  under  realistic  conditions  is 
badly  needed.  The  data  should  be  developed  using  the  coding  and  data 
development  procedures  that  will  be  used  in  an  operational  application.  The 
tests  conducted  thus  far  have  been  very  limited  In  scope.  They  have  been 
sufficient  to  demonstrate  the  feasibility  and  desirability  of  the  basic 
concepts.  However,  they  have  not  been  sufficient  In  scope  to  test  the  full 
range  of  problems  which  may  be  encountered  In  a  full-scale  application.  Such 
a  test  should  be  conducted  before  any  operational  application  of  an  automated 
technical  data  presentation  system  Is  made. 


Concluding  Remarks 

As  Indicated  earlier,  the  research  conducted  In  this  project  has 
demonstrated  that  automated  technical  data  presentation  systems  are  feasible. 
Further,  It  has  demonstrated  that  a  well-designed  system  will  be  accepted  and 
used  by  technicians.  In  fact,  the  technicians  who  participated  in  the  CMAS  II 
tests  were  eager  to  see  such  a  system  Implemented,  and  soon. 

Although  further  research  Is  needed  to  refine  the  technology  and  to 
develop  the  optimum  system,  the  technology  has  progressed  to  the  point  that 
the  development  of  applications  of  the  technology  to  operational  systems 
should  be  undertaken.  The  available  evidence  Indicates  that  such  a  system  has 
the  potential  to  greatly  Improve  the  efficiency  of  the  Air  Force  technical 
order  system  and  maintenance  operations. 
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GLOSSARY 


i 


ABDA 

ABDR 

AFHRL 

APS 

ATOS 

AMS 

CAMS 

CPB 

CRT 

EDM 

FEA 

FIND 

FPJPA 

FSCM 

FTD 

IF 

IFF 

IMIS 

IPB 

JGM 

LRU 

LTTA 

MANOVA 

MDC 

MDIS 

MMI 

NPRDC 

NTIPS 

PCMAS 

QA 

SGML 

SMR 

SRU 

TCTO 

TO 

UII 


Aircraft  Battle  Damage  Assessment 
Aircraft  Battle  Damage  Repair 
Air  Force  Human  Resources  Laboratory 
Authoring  and  Presentation  System 
Automated  Technical  Order  System 
Automated  Maintenance  System 
Core  Automated  Maintenance  System 
Consolidated  Parts  Breakdown 
Cathode  Ray  Tube 
Electronic  Display  Module 
Front-End  Analysis 

Fault  Identification  by  Nodal  Dependency 

Fully  Procedural i zed  Job  Performance  Aid 

Federal  Supply  Code  for  Manufacturers 

Field  Training  Detachment 

Intermediate  Frequency 

Identify  Friend  or  Foe 

Integrated  Maintenance  Information  System 

Illustrated  Parts  Breakdown 

Job  Guide  Manual 

Line  Replaceable  Unit 

Logic  Tree  Troubleshooting  Aid 

Multiple  Analysis  of  Variance 

Maintenance  Data  Collection 

Maintenance  Diagnostics  Information  System 

Man/Machine  Interface 

Navy  Personnel  Research  and  Development  Center 

Navy  Technical  Information  Presentation  System 

Portable  Computer-based  Maintenance  Aids  System 
Quality  Assurance 

Standard  Generalized  Markup  Language 
Source,  Maintenance,  and  Recoverability 
Shop  Repairable  Unit 
Time  Compliance  Technical  Order 
Technical  Order 
Unified  Industries,  Inc. 
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APPENDIX  A:  CMAS  II  USER'S  GUIDE 


The  Computer-based  Maintenance  Aids  System  II  (CMAS  II)  Is  a  prototype 
automated  system  for  the  presentation  of  technical  data  for  use  by  Air  Force 
maintenance  personnel.  The  system  has  been  designed  to  test  various  concepts 
for  the  presentation  of  technical  data  on  a  computer  display.  Evaluation  of 
the  system  will  provide  the  basis  for  the  design  of  a  system  for  operational 
use. 


SYSTEM  HARDWARE 

Grid  Compass  Computer,  Model  1139  Grid  Systems  10-MByte  Hard  Disk. 


TECHNICAL  DATA  CONTENTS: 

The  CMAS  II  demonstration  system  contains  the  data  required  for  the 
maintenance  of  the  RT-728A  Receiver-Transmitter.  The  information  has  been 
adapted  from  TO  12P4-2APX64-2  and  TO  12P4-APX64-4.  The  system  provides  the 
following  Information: 

Theory  of  Operation  (complete  unit) 

Checkout  and  Analysis  (selected  sections) 

Maintenance  Instructions  for  the  Coder  module  Including: 

Theory  of  Operation 

Remove  and  Replace  Procedures  (selected  procedures) 

Troubleshooting  Procedures  (to  circuit  board) 

Schematic  Diagrams  (each  circuit  board) 

Illustrated  Parts  Breakdown  (selected  portions) 


FEATURES 

Rapid  Location  of  Information.  The  system  provides  three  features  which 
allow  you  to  rapidly  locate  and  call  up  a  specific  piece  of  Information.  They 
are: 


Table  of  Contents:  Information  can  be  located  and  retrieved  by  selecting 
from  a  series  of  tables  of  contents. 

Direct  Access:  You  can  go  to  any  subsection  or  procedure  by  entering  the 
direct  access  mode  and  typing  In  the  exact  title.  Also,  If  you  know  the  frame 
number,  you  can  go  directly  to  that  frame  by  typing  the  number. 

Options  Menu.  Each  frame  has  an  associated  options  menu.  The  options 
menu  lists  specific  supplemental  Information  (such  as  a  schematic)  which  you 
may  need  to  complete  a  task.  For  example.  If  you  are  performing  a  checkout  of 
the  transmitter  module,  the  options  frame  will  take  you  directly  to  the 
transmitter  schematic  or  IPB  Information  for  the  transmitter.  The  options 
form  also  provides  an  option  to  go  to  the  table  of  contents  or  the  direct 
access  mode. 


i 

i 

i 
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Job  Guide  Format.  Instructions  for  Maintenance  procedures  are  presented 
in  a  Job  Guide  format.  Step-by-step  procedures  are  provided  for  each  task. 
Illustrations  of  the  referenced  hardware  are  provided  as  appropriate  for  the 
experience  level  of  the  user. 

Multiple  Levels  of  Detail.  The  system  provides  maintenance  instructions 
for  procedural  tasks  at  two  levels  of  detail  called  "tracks."  Nonprocedural 
information  such  as  theory  of  operation  is  presented  at  one  level  of  detail. 
The  levels  of  detail  are: 

Track  1 :  Instructions  presented  at  this  level  are  least  detailed.  They 
are  intended  for  use  by  technicians  who  work  on  the  system  daily  and  are  fully 
qualified  on  it.  The  procedures  provide  only  the  information  required  by  the 
experienced  technician  who  knows  the  system  well. 

Track  2:  Instructions  presented  at  this  level  are  the  most  detailed. 
These  instructions  are  designed  to  provide  the  inexperienced  technician  with 
all  of  the  Information  that  he  needs  to  do  the  task .  They  are  intended  for 
use  by  apprentice  technicians  with  limited  experience  or  by  experienced 
technicians  who  are  new  to  the  system  or  have  not  worked  on  the  system  for 
some  time. 

Pool  Information.  The  system  provides  for  quick  access  to  a  variety  of 
support  information  such  as  theory  of  operation,  schematic  drawings,  and  parts 
information. 

Beep.  The  computer  will  make  a  beeping  sound  If  you  enter  a  request  that 
It  cannot  accept .  This  will  happen  If:  (a)  you  press  the  wrong  key  (for 
example,  C  Instead  of  B),  (b)  you  make  a  selection  which  is  not  available  (for 
example,  you  press  the  space  bar  to  go  to  the  next  frame  but  the  next  frame 
option  Is  not  available),  or  (c)  you  enter  a  request  by  the  direct  access  mode 
for  information  that  Is  not  In  the  system  (for  example,  you  enter  the  number 
of  a  frame  that  does  not  exist). 

Scroll.  Some  drawings  and  illustrations  required  by  the  maintenance 
technician  are  too  large  to  appear  on  the  screen  at  one  time.  The  system 
overcomes  this  problem  by  providing  the  capability  to  pan  or  scroll  the 
illustration  so  that  the  drawing  may  be  "moved"  across  the  screen  to  view  a 
different  portion. 


PRESENTATION  FORMAT 

The  technical  data  are  organized  In  a  series  of  frames.  A  frame  is  the 
Information  presented  at  one  time  on  the  screen.  Although  several  types  of 
information  may  be  presented  In  a  frame,  there  are  four  pieces  of  information 
which  are  always  present  on  the  frame.  They  are  the  TO  number,  the  title  of 
the  data  presented,  the  frame  number,  and  prompts.  The  basic  layout  of  a 
typical  frame  is  shown  below. 
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Procedure  Title 


T  0  Nueber 


/ 

12P-2ARX-< 


Fri 


-64-2 


PRELIMINARY  CONTROL  SETTINGS  AMD  CONNECTIONS 


21  21  2C 


V. 


U. 


Set  AN/APH-239A  transponder  sat  control 
HAS TER  SHITCH  (1)  to  STBV. 

Verify  that  AN/APH239A  SWITCHED  AC  POWER 
light  (2)  is  on. 


Is  SWITCHED  AC  POWER 
light  on?  V  or  N 


>  0  o o  qq  li. 


APM-239A 


Procedural 

Question 


Prompts 


STARTING  THE  SYSTEM 
To  operate  the  system: 

1.  Turn  the  hard  disk  on.  The  switch  Is  located  on  the  back  of  the  unit 
In  the  upper  right  corner. 

2.  Turn  the  computer  on.  The  switch  Is  located  on  the  back  of  the 
computer  In  the  upper  right  corner. 

After  being  turned  on,  the  computer  requires  about  2  minutes  to  warm  up, 
perform  a  self-test,  and  load  the  required  software.  The  Introductory  frame 
will  appear  when  the  computer  Is  ready  for  operation. 


LOCATING  AND  RETRIEVING  INFORMATION  FROM  THE  SYSTEM 

The  CMAS  provides  two  basic  methods  of  locating  a  specific  piece  of 
Information.  Information  may  be  located  and  retrieved  by  using  a  series  of 
menus  or  by  using  the  direct  access  method. 

Table  of  Contents  Method  (menu  method): 

A  series  of  Tables  of  Contents  provide  the  basic  method  for  locating 
Information  In  the  CMAS  II.  To  locate  data  using  the  menu  method: 

1.  Use  the  space  bar  to  page  forward  to  the  main  Table  of  Contents. 

2.  Select  the  section  containing  the  desired  Information  from  the  Table 
of  Contents.  Enter  the  number  corresponding  to  the  selection  and  press  the 
code  and  return  keys  simultaneously.  (Note:  Enter  the  number  only.)  Do  not 
put  a  space  or  any  other  character  before  or  after  the  number.  If  you  attempt 


to  enter  a  number  not  in  the  aenu,  the  computer  will  respond  "Invalid  Request, 
Re-enter."  If  this  happens,  re-enter  the  number  and  press  code/return. 

3.  A  aenu  for  the  section  of  the  data  base  will  be  displayed.  Follow  the 
same  procedure  to  select  the  desired  Information. 

4.  At  this  point,  the  first  frame  of  the  desired  technical  data  will  be 
displayed,  or  a  third  aenu  will  appear.  The  third  aenu  provides  a  further 
breakdown  of  the  technical  information  in  that  section  from  which  you  may 
choose. 


Direct  Access  Method: 

The  direct  access  method  allows  you  to  go  directly  to  a  specific  section 
of  the  technical  data,  provided  that  you  know  the  exact  title  or  the  number  of 
the  first  frame.  It  also  allows  you  to  go  directly  to  any  specific  frame  of 
Information,  provided  that  you  know  the  frame  number.  This  feature  Is  useful 
if  you  stop  work  in  the  middle  of  a  procedure  and  want  to  restart  at  the  same 
place  at  a  later  time.  To  use  the  direct  access  method: 

From  start-up: 

1.  Use  the  space  bar  to  page  to  the  title  page.  Press  D. 

2.  The  direct  access  frame  will  appear.  Type  the  name  of  the  procedure 
or  the  number  of  the  frame  desired  In  the  box  and  press  the  CODE  and  RETURN 
keys  simultaneously.  The  name  or  frame  number  must  be  exactly  correct.  Do 
not  add  spaces  or  other  characters  before,  after,  or  within  the  section  title 
or  number.  If  your  entry  Is  Incorrect  or  the  specified  section  or  frame  number 
does  not  exist  in  the  data  base,  the  computer  will  respond  "Invalid  Request, 
Re-enter."  It  will  also  give  a  "beeping*  sound.  If  this  happens,  try  again. 
If  you  make  a  typing  mistake,  use  the  backspace  key  to  erase  the  characters 
you  have  typed. 

From  within  the  data  base: 

1.  Press  the  "OP"  key  to  select  the  options  frame.  Select 
"Direct  Access  Mode." 

2.  Follow  the  procedure  described  In  2  above. 


USING  THE  SYSTEM 

The  CMAS  II  has  been  designed  for  simple  operation.  The  following 
features  are  provided  to  simplify  Its  use: 

Prompt  Line.  The  bottom  line  of  each  frame  provides  prompts  which  tell 
the  user  which  options  are  available.  The  usual  options  are: 

Next  *  Space  Bar.  If  you  press  the  space  bar,  the  system  will  display  the 
next  logical  frame. 


Back  *  B.  If  you  press  B,  the  system  Mill  display  the  previous  frame. 

Option  *  0.  If  you  press  0,  the  system  Mill  go  to  the  options  frame. 

Return  *  R.  This  prompt  Mill  appear  only  if  you  have  branched  from  a 
procedure  or  subsection  to  another  part  of  the  data  base  (such  as  to  a 
schematic}.  If  you  press  R,  the  system  will  display  the  frame  from  which  you 
branched  to  the  current  frame. 

Note:  You  will  note  that  some  frames  do  not  have  an  option  of  going  to 
the  next  frame.  The  choice  of  the  next  frame  Is  dependent  upon  the  answer  to 
a  question  asked  In  the  text  of  the  frame.  In  this  case,  the  question  and  the 
options  are  listed  In  the  frame.  Answering  the  question  will  take  you  to  the 
next  logical  frame. 

Scroll.  The  scroll  feature  Is  used  when  a  drawing  or  illustration  Is  too 
large  to  fit  on  the  screen.  When  the  drawing  is  too  large  and  scrolling  is 
required,  the  scroll  capability  Is  made  available  to  the  user.  When  the 
scroll  capability  Is  active,  the  following  symbol  will  appear: 

The  illustration  Is  scrolled  by  using  the  arrow  keys  located  on  the  right 
side  of  the  keyboard.  Pressing  the  left  arrow  causes  the  drawing  to  move  to 
the  left.  Pressing  the  up  arrow  causes  the  drawing  to  move  upward.  Each 
press  of  an  arrow  key  will  move  the  schematics  about  1/4  inch.  If  you  press 
the  code  key  and  the  arrow  key  at  the  same  time,  the  schematic  will  move  about 
1/10  inch. 

Scrolling  Is  accomplished  by  using  the  arrow  keys  on  the  right  side  of  the 
keyboard. 


Illustrated  Parts  Breakdown  (IPB). 

The  CMAS  II  Is  designed  to  provide  rapid  access  to  parts  information.  Two 
methods  are  provided  for  retrieving  IPB  Information.  This  Information  can  be 
obtained  through  the  direct  access  method  If  you  know  the  part  number  or 
reference  designator  number.  If  you  do  not  know  the  part  number  or  reference 
designator,  you  can  locate  the  part  on  an  illustration.  To  retrieve  IPB 
Information: 

From  the  main  Table  of  Contents: 

1.  Select  Illustrated  Parts  Breakdown  from  the  Table  of  Contents. 

2.  The  IPB  Table  of  Contents  will  appear. 

3.  If  you  know  the  part  number  or  reference  designator,  select  the  direct 
parts  Information  mode.  If  you  do  not  know  the  part  number  or  reference 
designator  or  wish  to  view  the  Illustration,  select  the  appropriate  section  of 
the  unit. 

4.  If  you  use  the  direct  parts  Information  approach,  the  system  will 
display  an  illustration  of  the  equipment  and  a  listing  of  the  pertinent 
Information  on  the  subject  part. 
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5.  If  you  do  not  use  the  direct  parts  information  approach,  the  system 
will  display  an  illustration  of  the  equipment.  Two  techniques  are  used  to 
permit  you  to  select  information  on  specific  parts  Illustrated.  In  some 
cases,  access  to  specific  Information  is  provided  from  Index  numbers  keyed  to 
the  Illustration.  In  other  cases,  a  part  Is  selected  by  moving  a  cursor  arrow 
to  the  part  of  Interest  on  the  drawing  and  pressing  code  return.  The  cursor 
arrow  always  starts  at  the  lower  right  corner  of  the  drawing.  One  press  of 
the  cursor  will  move  the  cursor  about  1/4  Inch.  If  you  press  the  code  and 
arrow  key  at  the  same  time,  the  cursor  will  move  about  1/10  Inch.  The  system 
provides  specific  directions  for  each  request. 


EXERCISES  FOR  THE  CMAS  II  USER'S  GUIDE 

Using  the  Table  of  Contents  method,  find  the  Theory  of  Operation  explanation 
of  Pulse  Designations. 

Using  the  Table  of  Contents  method,  find  the  preliminary  operational  check. 

Using  the  Table  of  Contents  method,  find  the  Remove  and  Replace  Instructions 
for  the  Coder  Module. 

Using  the  Direct  Access  method,  find  the  Emergency  Enable  Check  section  of  the 
Checkout  Procedure.  Call  up  the  Options  Form. 

Use  the  Direct  Access  Method  to  find  frame  2.8.3C.  Use  the  Options  Form  to 
select  More  Detail. 

From  the  Main  Menu,  select  Illustrated  Parts  Breakdown.  From  the  IPB  Menu, 
select  Direct  Parts  Information. 

Using  the  Direct  Parts  Information  method,  find  the  part  number  for  Reference 
Designator  A5A1Q4. 

Using  the  Direct  Parts  Information  method,  find  the  reference  designator  for 
Part  Number  01A236508. 

Using  the  Direct  Access  Method,  select  the  Troubleshooting  Procedures. 
Perform  the  following: 

1.  From  the  first  frame,  select  Options  Form. 

2.  From  the  Options  Form,  select  Schematics. 

3.  From  the  Schematics  Menu,  select  the  Coder  Module. 

4.  From  the  Coder  Module,  select  board  A5A1. 

5.  Use  the  arrow  keys  to  scroll  the  schematic. 

6.  Locate  the  "MODE  C  A  Enable." 


160 


APPENOIX  B:  CMAS  II  DEMONSTRATION  OUTBRIEFS 


Airman  First  Class  -  Low  Experience 

1.  I  want  It  to  stay  here.  It  Is  a  lot  faster  than  using  the  TO.  You 
don't  have  to  fumble  through  pages.  You  can  go  to  exactly  what  you  want. 

2.  You  can  find  what  you  want  right  away.  It  Is  easier  to  read.  The  TO 
Is  harder  to  use  because  you  lose  your  place  and  the  print  Is  smaller.  Ulth 
the  computer  you  can  tell  exactly  where  you  are.  It  must  be  easier  to  use 
because  of  the  way  It  Is  displayed,  because  the  data  looks  like  It  came  right 
from  the  TO. 

3.  I  really  like  the  way  the  IPB  works.  I  know  how  much  time  It  would 
save  me  because  It  Is  my  job  to  order  parts. 

4.  There  Isn't  anything  I  don't  Tike  about  It.  It  Is  easy  to  operate  and 
work  from.  What  Is  there  to  not  like?  There  Isn't  anything  about  It  that  Is 
confusing. 

5.  One  advantage  to  the  TO  schematic  Is  that  you  can  look  at  the  whole 
thing  at  one  time.  I  don't  see  any  way  out  of  It.  If  you  made  them  small 
enough  to  fit  on  the  screen,  you  wouldn't  be  able  to  read  them.  If  you  made 
the  screen  big  enough.  It  would  be  too  big. 


Staff  Sergeant  (Quality  Assurance  Inspector)  -  High  Experience 

1.  It  simplifies  things  and  Is  easy  to  use. 

2.  Only  question  Is  more  detail.  A  technician  can  think  he  knows 
everything  and  leave  a  step  out.  Questions  whether  or  not  technician  should 
have  option  of  choosing  less  detailed  level.  I  left  a  cable  off  using  less 
detail. 

3.  This  would  force  people  to  use  TOs  more.  They  can  open  a  book  and 
leave  It  to  the  right  page,  but  If  the  computer  stays  on  one  frame  very  long, 
they're  not  using  It  and  supervisors  can  spot  It  quickly. 

4.  Schematics  were  unclear  around  bottom  edges.  Mere  too  crunched  up  to 
really  tell  where  a  signal  or  test  point  was. 

5.  Don't  feel  that  Instructions  on  using  test  equipment  (oscilloscope, 
etc.)  should  be  part  of  regular  checkout.  Should  have  separate  cartridges  (or 
disks  or  whatever)  on  how  to  use  different  test  equipment  available  to  teach 
with,  but  anyone  working  on  the  system  should  already  know  how  to  use  the 
equipment. 

6.  Mould  be  very  helpful  to  have  automatic  Interconnection  between 
related  schematics.  Scroll  off  one  and  onto  the  next  appropriate  one  for 
desired  signals. 

7.  Two  direct  access  methods  were  confusing.  Didn't  always  know  which 
one  you  were  In. 
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8.  Liked  troubleshooting  procedures  best  of  all.  There  Is  a  real  need 
for  accurate  troubleshooting  after  determining  which  module  Is  bad. 

9.  Would  definitely  like  to  see  similar  system  Incorporated  Into  USAF. 
System  would  help  a  person  become  a  better  troubleshooter  and  technician  If 
the  person  wanted  to  learn  from  the  system  and  not  be  a  slave  to  the 
computer.  Training  and  normal  troubleshooting  procedures  should  not  be 
together  In  the  same  system. 


Airman  First  Class  -  High  Experience 

1.  With  practice  It  would  be  great.  The  IPB  Is  real  nice.  I  like  It  a 

lot. 

2.  It  Is  hard  to  find  things  on  the  schematic.  With  the  TO  you  can  look 
at  the  whole  thing.  You  could  get  the  job  done  with  the  computer  but  It 
wouldn't  be  as  convenient. 

3.  The  computer  Is  easy  to  use.  You  can  use  It  with  just  a  few  minutes 
practice  without  any  problem  at  all.  I've  never  had  any  experience  with 
computers  and  I  can't  type  but  I  didn't  have  any  problem  using  It. 

4.  The  way  that  Information  Is  presented  makes  It  easy  to  use.  The 
checklist  layout  with  the  different  levels  makes  It  easy  to  read.  The 
drawings  which  point  out  the  location  of  knobs  and  connectors  make  It  easy  to 
use. 

5.  I  would  like  to  have  more  Items  per  frame.  Just  having  a  few  Items 
makes  It  a  little  slow. 

6.  It  Is  pretty  convenient  to  use.  It  Isn't  much  bigger  than  the  TO.  It 
Is  easy  to  get  lost  on  a  TO  page.  With  the  computer,  everything  Is  right 
there.  You  can't  get  lost.  This  makes  It  real  nice. 

7.  I  would  like  to  see  the  system  have  battery  power. 

8.  I  would  like  to  see  It  Implemented  In  the  Air  Force.  It  would  be 

great  for  changes. 


Airman  First  Class  -  Low  Experience 

1.  I  like  It.  I  have  a  very  positive  reaction  to  It.  You  can  get 
Information  a  lot  faster  once  you  become  acquainted  with  the  system.  It  sure 
beats  flipping  through  the  pages  of  a  TO. 

2.  It  would  be  very  easy  to  make  changes  to  the  technical  data  If  there 
Is  anything  wrong  with  the  Information  In  the  system. 

3.  Once  the  bugs  are  out.  It  will  be  a  very  good  troubleshooting  aid.  It 
does  In  fact  tell  you  what  to  do  next.  It  would  speed  up  your  troubleshooting 
time.  I  don't  know  If  that  Is  good  or  bad.  It  would  make  troubleshooting  so 
simple  that  anyone  could  do  It  with  limited  training.  I  guess  It  would  be 
good  for  the  Air  Force. 
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4.  The  system  could  have  a  lot  larger  screen  for  reading  schematics.  It 
would  give  more  detail  and  make  it  easier  to  l&ate  things  on  the  schematic. 
A  12  to  14  Inch  diagonal  screen  mould  sure  help.  You  don't  need  a  larger 
screen  except  for  reading  schematics.  The  screen  on  this  computer  is  big 
enough  for  everything  else. 

5.  The  screen  is  easily  readable. 

6.  I  can't  think  of  anything  else  I  have  against  It.  All  in  all  It  Is  a 
pretty  good  system. 

7.  It  Is  great  for  looking  up  part  numbers.  Just  to  be  able  to  put  all 
of  the  parts  Information  on  the  screen  so  quickly  Is  Incredible.  It  saves  a 
good  5  to  10  minutes. 

8.  I  would  like  to  see  It  Implemented  In  the  Air  Force.  I  think  we  will 
always  have  TOs  In  book  form.  There  will  always  be  books  sitting  on  the 
shelf.  Computers  can  lose  Information.  It  might  be  possible  to  replace  TOs 
with  a  computer  If  there  Is  a  way  to  keep  from  losing  Information  (such  as 
from  magnetic  fields). 


Airman  First  Class  -  High  Experience 


1.  The  computer  is  very  fast.  It  Is  much  easier  to  use  than  the  TO. 
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2.  When  you  go  through  the  checkout,  you  must  read  every  step.  It  Is 
easy  to  skip  over  a  step  In  the  TO.  With  the  computer  you  are  not  as  likely 
to  skip  over  a  step. 

3.  I  like  being  able  to  go  to  the  table  of  contents  and  go  anywhere  (In 
the  data  base)  I  want  to  go.  It  Is  easy  to  get  where  you  want  to  go.  If  I 
used  the  computer  a  lot,  I  probably  wouldn't  get  lost.  It  Is  easy  to  move 
around  and  get  where  you  want  to  go. 


4.  The  graphics  were  really  neat.  The  way  they  point  to  a  component 
makes  It  easier  to  find  things. 

5.  Using  the  computer  was  fun  -  a  lot  more  Interesting  than  the  stupid  TO. 


n 


6.  I  didn't  like  the  schematics.  It  Is  neat  to  be  able  to  scroll  but  It 

Is  not  like  having  the  whole  thing  In  front  of  you.  You  could  use  the 

schematics  after  awhile  though. 

7.  I  would  like  to  see  the  computer  put  on  a  caddy  like  you  use  for  an 
0~scope  so  that  I  could  set  It  beside  me  and  move  It  wherever  I  want  to. 

8.  The  screen  needs  a  tilt  lock  to  keep  It  In  place. 

9.  I  would  like  to  see  the  computer  Implemented  in  the  Air  Force.  I 

thought  it  was  a  joke  at  first  but  now  I  think  It  will  work.  I  liked  using  It. 
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Airman  First  Class  -  Low  Experience 

1.  I  like  it  a  whole  lot  better  than  TOs.  It  is  a  whole  lot  easier  not 
to  skip  steps  with  the  computer.  It  is  easier  because  you  don't  get  lost.  A 
lot  of  times  the  TO  page  will  flip  while  you  are  doing  something.  Then  you 
have  to  find  where  you  were  again.  With  the  computer  you  stay  right  where  you 
were. 

2.  I  like  the  IPB  a  lot.  It  saves  a  lot  of  time.  Especially  when  you  go 
to  get  information  on  a  component.  You  have  everything  you  need  right  there 
and  don't  have  to  go  to  a  file  (microfiche  file)  to  get  part  of  it. 

3.  The  notes  and  cautions  are  good.  A  lot  of  times  you  tend  to  skip  over 
them  in  the  TO.  If  they  are  on  the  screen,  you  go  ahead  and  read  them. 

4.  I  liked  having  both  levels  of  detail.  I  used  both  levels  but 
preferred  the  more  detailed  level.  I  liked  the  Illustrations  too.  They  tell 
you  exactly  where  to  put  the  connections.  They  are  especially  good  for  people 
who  have  not  gone  through  the  procedure  before. 

5.  I  liked  the  troubleshooting  too.  It  is  a  whole  lot  more  specific. 
Plus,  It  worked. 

6.  There  Isn't  anything  that  I  didn't  like  about  It.  I  don't  have  any 
suggestions  for  redesigning  It. 

7.  I  would  like  to  see  it  implemented  in  the  Air  Force. 


Staff  Sergeant  (FTD  Instructor)  -  High  Experience 

1.  There  Is  a  world  of  difference  between  the  computer  and  the  TO. 

2.  The  computer  Is  so  easy  to  use.  It  formats  things  and  limits  the 
amount  of  Information  that  you  see  at  a  time,  which  limits  the  confusion.  It 
puts  information  In  a  step-by-step  format  which  is  easy  to  follow. 

3.  With  the  TO  you  have  the  problem  of  the  pages  being  open  and  having  to 
look  back  and  forth  and  maybe  missing  a  step  in  the  procedure.  Mlth  this  it  Is 
Impossible,  almost. 

4.  The  IPB  is  an  outstanding  example  of  what  the  computer  can  do.  You  can 
put  in  a  part  number  and  It  spits  the  Information  right  out  to  you.  That's 
excellent. 

5.  It  could  be  extremely  useful  in  training.  The  procedures  outline  the 
process  logically.  The  simplicity  of  the  Information  makes  it  Ideal  for 
training.  The  way  Information  Is  formatted  in  the  TO  you  have  to  go  from 
section  to  section  and  try  to  find  It. 

6.  I  especially  liked  the  operational  checkout  and  the  IPB. 

7.  At  first  I  didn't  like  the  schematics  because  they  show  such  a  limited 
view  but  after  a  little  familiarization  It  seems  that  they  may  not  be  as  good 
as  the  TO  but  they  are  sufficient. 


8.  I  don't  see  anything  I  could  do  to  improve  it. 


9.  As  an  instructor  and  a  technician  I  think  the  system  is  ideal.  It 
facilitates  much  quicker  and  more  precise  maintenance.  I  kind  of  regret  that 
it  can't  be  here  tomorrow.  It  is  an  ideal  way  to  go. 


Senior  Airman  (Inertial  Navigation  Shop)  -  No  Experience 

1.  I  thought  It  was  pretty  easy  to  use.  The  program  setup  itself  I 
thought  was  fantastic. 

2.  I  thought  it  was  pretty  easy  following  the  procedure  on  the  computer. 
There  were  some  parts  I  had  trouble  with,  but  it  didn't  take  long  to  figure 
out  how  to  do  it  by  going  to  the  schematics  and  troubleshooting  procedures. 
It  is  a  real  easy  system  to  use. 

3.  I  liked  the  scrolling  on  the  schematics.  I  think  It  was  a  lot  faster 
than  trying  to  use  the  TO  where  you  have  to  pull  out  the  pages.  It  was  easier 
to  use  than  the  TO.  It  was  easy  to  push  the  button  and  trace  the  wiring  as  it 
flows  by.  You  don't  get  lost  tracing  a  signal. 

4.  It  is  an  easy  system. 

5.  This  was  totally  a  lot  faster  than  the  TO.  This  will  expedite 
troubleshooting  and  system  checkout. 

6.  I  went  through  the  theory  of  operation  and  got  a  general  Idea.  If  I 
had  been  reading  the  TO,  I  probably  would  have  been  bored  to  death.  I  didn't 
get  tired  reading  or  anything. 

7.  I  sure  would  like  to  see  it  implemented  in  the  Air  Force. 
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